Qwen 3.5 122B-A10B (MoE)
Qwen 3.5 122B-A10B (MoE) needs roughly 85.6 GB VRAM at Q4_K_M quantization (275.7 GB at FP16). 29 GPUs we track can run it fully in VRAM at 8k context.
29 GPUs run this natively · 10 with CPU offload
Qwen 3.5 122B-A10B (MoE) is a Mixture of Experts (MoE) model with 122B total parameters but only 10B active per token developed by Alibaba. February 2026 flagship MoE from the Qwen3.5 line — 122B total parameters, 10B active, 262K context with vision support, Apache 2.0 licensed.
To run Qwen 3.5 122B-A10B (MoE) locally: Q3_K needs roughly 58-60GB — an 80GB GPU or a 64GB+ unified-memory Mac is the practical minimum; it won't fit on a single 48GB card at this quantization. The 10B active-parameter count keeps decode speed reasonable once loaded. As a MoE model, inference speed depends on active parameters (10B) rather than total size.
MMLU-Pro 86.7 is the highest in the Qwen3.5 family and competitive with other 2026-era frontier open-weight releases.
VRAM at each quantization
Figures below assume 8k context; KV cache grows linearly as context length increases.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 488.0 GB | 2.15 GB | 549.0 GB |
| BF16 | 244.0 GB | 2.15 GB | 275.7 GB |
| FP16 | 244.0 GB | 2.15 GB | 275.7 GB |
| Q8_0 | 129.7 GB | 2.15 GB | 147.7 GB |
| Q6_K | 100.2 GB | 2.15 GB | 114.6 GB |
| Q5_K_M | 86.9 GB | 2.15 GB | 99.7 GB |
| Q4_K_M | 74.3 GB | 2.15 GB | 85.6 GB |
| Q3_K_Mrec | 58.7 GB | 2.15 GB | 68.1 GB |
| Q2_K | 46.5 GB | 2.15 GB | 54.5 GB |
| NVFP4cuda | 61.0 GB | 2.15 GB | 70.7 GB |
KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Qwen 3.5 122B-A10B (MoE) natively (29)
- NVIDIA H100 80GBNVFP4 · 115.7 t/s
- NVIDIA A100 80GBNVFP4 · 70.4 t/s
- NVIDIA RTX Pro 6000NVFP4 · 46.4 t/s
- NVIDIA DGX Spark (128GB)NVFP4 · 9.4 t/s
- AMD Instinct MI300XQ8_0 · 91.7 t/s
- AMD Strix Halo (128GB)Q6_K · 5.6 t/s
- AMD Strix Halo (96GB)Q4_K_M · 7.4 t/s
- AMD Strix Halo (64GB)Q2_K · 11.2 t/s
- Apple M5 Max (128GB)Q6_K · 16.6 t/s
- Apple M5 Max (64GB)Q2_K · 33.1 t/s
- Apple M4 Ultra (384GB)BF16 · 12.7 t/s
- Apple M4 Ultra (192GB)Q8_0 · 23.2 t/s
- Apple M4 Max (128GB)Q6_K · 14.8 t/s
- Apple M4 Max (96GB)Q4_K_M · 19.5 t/s
- Apple M4 Max (64GB)Q2_K · 29.4 t/s
- Apple M3 Ultra (512GB)BF16 · 9.5 t/s
- Apple M3 Ultra (256GB)Q8_0 · 17.4 t/s
- Apple M3 Ultra (96GB)Q4_K_M · 29.2 t/s
- Apple M3 Max (128GB)Q6_K · 10.8 t/s
- Apple M3 Max (96GB)Q4_K_M · 14.3 t/s
- Apple M3 Max (64GB)Q2_K · 21.6 t/s
- Apple M2 Ultra (384GB)BF16 · 9.3 t/s
- Apple M2 Ultra (192GB)Q8_0 · 17 t/s
- Apple M2 Max (96GB)Q4_K_M · 14.3 t/s
- Apple M2 Max (64GB)Q2_K · 21.6 t/s
- Apple M1 Ultra (128GB)Q6_K · 21.7 t/s
- Apple M1 Ultra (64GB)Q2_K · 43.1 t/s
- Apple M1 Max (64GB)Q2_K · 21.6 t/s
- Intel Data Center GPU Max 1550Q6_K · 72.1 t/s
Plus 10 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 5090Q2_K · 4.3 t/s
- NVIDIA A100 40GBQ2_K · 7.1 t/s
- NVIDIA L40SNVFP4 · 4.3 t/s
- NVIDIA RTX A6000NVFP4 · 4.2 t/s
- NVIDIA RTX 5000 AdaQ2_K · 4 t/s
- NVIDIA RTX 6000 AdaNVFP4 · 4.3 t/s
- AMD Radeon PRO W7800Q2_K · 4 t/s
- AMD Radeon PRO W7900Q3_K_M · 4.8 t/s
- AMD Radeon AI Pro 9700 32GBQ2_K · 4.1 t/s
- Intel Data Center GPU Max 1100Q3_K_M · 5 t/s
Continue reading
Frequently asked questions
- What are the VRAM requirements for Qwen 3.5 122B-A10B (MoE)?
- Qwen 3.5 122B-A10B (MoE) requires approximately 85.6 GB of VRAM at Q4_K_M quantization, 147.7 GB at Q8, and 275.7 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Qwen 3.5 122B-A10B (MoE) have?
- Qwen 3.5 122B-A10B (MoE) has 122 billion total parameters, but only 10 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is Qwen 3.5 122B-A10B (MoE)?
- Qwen 3.5 122B-A10B (MoE) achieves an MMLU-Pro score of 86.7, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
- Can Qwen 3.5 122B-A10B (MoE) run on a 16 GB GPU?
- No. At Q4_K_M, Qwen 3.5 122B-A10B (MoE) needs 85.6 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Qwen 3.5 122B-A10B (MoE) run on a 24 GB GPU?
- No. Even at Q4_K_M, Qwen 3.5 122B-A10B (MoE) needs 85.6 GB. Consider a multi-GPU server with 80 GB+ total VRAM.
- What is the smallest quantization for Qwen 3.5 122B-A10B (MoE) that fits in 24 GB of VRAM?
- Qwen 3.5 122B-A10B (MoE) cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 54.5 GB at Q2_K.
- What GPU do I need to run Qwen 3.5 122B-A10B (MoE) locally?
- You need a multi-GPU server. At Q4_K_M, Qwen 3.5 122B-A10B (MoE) needs 85.6 GB VRAM, more than any single consumer GPU. Consider 2–4× H100 or A100 GPUs.