Support Models on Ascend NPU#
This section describes the models supported on the Ascend NPU, including Large Language Models, Multimodal Language Models, Embedding Models, Reward Models and Rerank Models. Mainstream DeepSeek/Qwen/GLM series are included. You are welcome to enable various models based on your business requirements.
Large Language Models#
Models |
Model Family |
A2 Supported |
A3 Supported |
|---|---|---|---|
DeepSeek V3/V3.1 |
DeepSeek |
√ |
√ |
DeepSeek-V3.2-W8A8 |
DeepSeek |
√ |
√ |
DeepSeek-R1-0528-W8A8 |
DeepSeek |
√ |
√ |
DeepSeek-V2-Lite-W8A8 |
DeepSeek |
√ |
√ |
Qwen/Qwen3-30B-A3B-Instruct-2507 |
Qwen |
√ |
√ |
Qwen/Qwen3-32B |
Qwen |
√ |
√ |
Qwen/Qwen3-0.6B |
Qwen |
√ |
√ |
Qwen3-235B-A22B-W8A8 |
Qwen |
√ |
√ |
Qwen/Qwen3-Next-80B-A3B-Instruct |
Qwen |
√ |
√ |
Qwen3-Coder-480B-A35B-Instruct-w8a8-QuaRot |
Qwen |
√ |
√ |
Qwen/Qwen2.5-7B-Instruct |
Qwen |
√ |
√ |
QWQ-32B-W8A8 |
Qwen |
√ |
√ |
meta-llama/Llama-4-Scout-17B-16E-Instruct |
Llama |
√ |
√ |
AI-ModelScope/Llama-3.1-8B-Instruct |
Llama |
√ |
√ |
LLM-Research/llama-2-7b |
Llama |
√ |
√ |
LLM-Research/Llama-3.2-1B-Instruct |
Llama |
√ |
√ |
mistralai/Mistral-7B-Instruct-v0.2 |
Mistral |
√ |
√ |
google/gemma-3-4b-it |
Gemma |
√ |
√ |
microsoft/Phi-4-multimodal-instruct |
Phi |
√ |
√ |
allenai/OLMoE-1B-7B-0924 |
OLMoE |
√ |
√ |
stabilityai/stablelm-2-1_6b |
StableLM |
√ |
√ |
CohereForAI/c4ai-command-r-v01 |
Command-R |
√ |
√ |
huihui-ai/grok-2 |
Grok |
√ |
√ |
ZhipuAI/chatglm2-6b |
ChatGLM |
√ |
√ |
Shanghai_AI_Laboratory/internlm2-7b |
InternLM 2 |
√ |
√ |
LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct |
ExaONE 3 |
√ |
√ |
xverse/XVERSE-MoE-A36B |
XVERSE |
√ |
√ |
HuggingFaceTB/SmolLM-1.7B |
SmolLM |
√ |
√ |
ZhipuAI/glm-4-9b-chat |
GLM-4 |
√ |
√ |
XiaomiMiMo/MiMo-7B-RL |
MiMo |
√ |
√ |
arcee-ai/AFM-4.5B-Base |
Arcee AFM-4.5B |
√ |
√ |
Howeee/persimmon-8b-chat |
Persimmon |
√ |
√ |
inclusionAI/Ling-lite |
Ling |
√ |
√ |
ibm-granite/granite-3.1-8b-instruct |
Granite |
√ |
√ |
ibm-granite/granite-3.0-3b-a800m-instruct |
Granite MoE |
√ |
√ |
AI-ModelScope/dbrx-instruct |
DBRX (Databricks) |
√ |
√ |
baichuan-inc/Baichuan2-13B-Chat |
Baichuan 2 (7B, 13B) |
√ |
√ |
baidu/ERNIE-4.5-21B-A3B-PT |
ERNIE-4.5 (4.5, 4.5MoE series) |
√ |
√ |
OpenBMB/MiniCPM3-4B |
MiniCPM (v3, 4B) |
√ |
√ |
moonshotai/Kimi-K2-Thinking |
Kimi |
√ |
√ |
eigen-ai-labs/gpt-oss-120b-bf16 |
GPTOSS |
√ |
√ |
allenai/OLMo-2-1124-7B-Instruct |
OLMo |
√ |
√ |
cyankiwi/MiniMax-M2-BF16 |
MiniMax-M2 |
√ |
√ |
upstage/SOLAR-10.7B-Instruct-v1.0 |
Solar |
√ |
√ |
bigcode/starcoder2-7b |
StarCoder2 |
√ |
√ |
arcee-ai/Trinity-Mini |
Trinity (Nano, Mini) |
√ |
√ |
Multimodal Language Models#
Models |
Model Family (Variants) |
A2 Supported |
A3 Supported |
|---|---|---|---|
Qwen/Qwen2.5-VL-3B-Instruct |
Qwen-VL |
√ |
√ |
Qwen/Qwen2.5-VL-72B-Instruct |
Qwen-VL |
√ |
√ |
Qwen/Qwen3-VL-30B-A3B-Instruct |
Qwen-VL |
√ |
√ |
Qwen/Qwen3-VL-8B-Instruct |
Qwen-VL |
√ |
√ |
Qwen/Qwen3-VL-4B-Instruct |
Qwen-VL |
√ |
√ |
Qwen/Qwen3-VL-235B-A22B-Instruct |
Qwen-VL |
√ |
√ |
deepseek-ai/deepseek-vl2 |
DeepSeek-VL2 |
√ |
√ |
deepseek-ai/Janus-Pro-1B |
Janus-Pro (1B, 7B) |
√ |
√ |
deepseek-ai/Janus-Pro-7B |
Janus-Pro (1B, 7B) |
√ |
√ |
openbmb/MiniCPM-V-2_6 |
MiniCPM-V / MiniCPM-o |
√ |
√ |
openbmb/MiniCPM-o-2_6 |
MiniCPM-V / MiniCPM-o |
√ |
√ |
google/gemma-3-4b-it |
Gemma 3 (Multimodal) |
√ |
√ |
mistralai/Mistral-Small-3.1-24B-Instruct-2503 |
Mistral-Small-3.1-24B |
√ |
√ |
microsoft/Phi-4-multimodal-instruct |
Phi-4-multimodal-instruct |
√ |
√ |
XiaomiMiMo/MiMo-VL-7B-RL |
MiMo-VL (7B) |
√ |
√ |
AI-ModelScope/llava-v1.6-34b |
LLaVA (v1.5 & v1.6) |
√ |
√ |
lmms-lab/llava-next-72b |
LLaVA-NeXT (8B, 72B) |
√ |
√ |
lmms-lab/llava-onevision-qwen2-7b-ov |
LLaVA-OneVision |
√ |
√ |
moonshotai/Kimi-VL-A3B-Instruct |
Kimi-VL (A3B) |
√ |
√ |
ZhipuAI/GLM-4.5V |
GLM-4.5V (106B) |
√ |
√ |
LLM-Research/Llama-3.2-11B-Vision-Instruct |
Llama 3.2 Vision (11B) |
√ |
√ |
rednote-hilab/dots.ocr |
DotsVLM-OCR |
√ |
√ |
Embedding Models#
Models |
Model Family |
A2 Supported |
A3 Supported |
|---|---|---|---|
intfloat/e5-mistral-7b-instruct |
E5 (Llama/Mistral based) |
√ |
√ |
iic/gte_Qwen2-1.5B-instruct |
GTE-Qwen2 |
√ |
√ |
Qwen/Qwen3-Embedding-8B |
Qwen3-Embedding |
√ |
√ |
Alibaba-NLP/gme-Qwen2-VL-2B-Instruct |
GME (Multimodal) |
√ |
√ |
AI-ModelScope/clip-vit-large-patch14-336 |
CLIP |
√ |
√ |
BAAI/bge-large-en-v1.5 |
BGE |
√ |
√ |
Reward Models#
Models |
Model Family |
A2 Supported |
A3 Supported |
|---|---|---|---|
Skywork/Skywork-Reward-Llama-3.1-8B-v0.2 |
Llama3.1 Reward |
√ |
√ |
Shanghai_AI_Laboratory/internlm2-7b-reward |
InternLM 2 Reward |
√ |
√ |
Qwen/Qwen2.5-Math-RM-72B |
Qwen2.5 Reward - Math |
√ |
√ |
Howeee/Qwen2.5-1.5B-apeach |
Qwen2.5 Reward - Sequence |
√ |
√ |
AI-ModelScope/Skywork-Reward-Gemma-2-27B-v0.2 |
Gemma 2-27B Reward |
√ |
√ |
Rerank Models#
Models |
Model Family |
A2 Supported |
A3 Supported |
|---|---|---|---|
BAAI/bge-reranker-v2-m3 |
BGE-Reranker |
√ |
√ |