Qwen 3.5 122B-A10B (MoE)
Qwen 3.5 122B-A10B (MoE) needs roughly 83.4 GB VRAM at Q4_K_M quantization (273.5 GB at FP16). 33 GPUs we track can run it fully in VRAM at 8k context.
33 GPUs run this natively · 11 with CPU offload
Qwen 3.5 122B-A10B (MoE) is a Mixture of Experts (MoE) model with 122B total parameters but only 10B active per token developed by Alibaba. February 2026 flagship MoE from the Qwen3.5 line: 122B total parameters, 10B active, 262K context with vision support, Apache 2.0 licensed.
To run Qwen 3.5 122B-A10B (MoE) locally: Q4_K_M needs roughly 74.3GB of weights, about 83.4GB total at a short context; the smallest single GPU this site tracks that fits it is a 96GB NVIDIA RTX Pro 6000, past an 80GB card. The 10B active-parameter count keeps decode speed reasonable once loaded. As a MoE model, inference speed depends on active parameters (10B) rather than total size.
MMLU-Pro 86.7 is the highest in the Qwen3.5 family and competitive with other 2026-era frontier open-weight releases.
VRAM at each quantization
Figures below assume 8k context. This model's hybrid attention stack caches only some layers, so KV cache grows much slower than linearly as context increases.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 488.0 GB | 0.20 GB | 546.8 GB |
| BF16 | 244.0 GB | 0.20 GB | 273.5 GB |
| FP16 | 244.0 GB | 0.20 GB | 273.5 GB |
| Q8_0 | 129.7 GB | 0.20 GB | 145.5 GB |
| Q6_K | 100.2 GB | 0.20 GB | 112.4 GB |
| Q5_K_M | 86.9 GB | 0.20 GB | 97.5 GB |
| Q4_K_Mrec | 74.3 GB | 0.20 GB | 83.4 GB |
| Q3_K_M | 58.7 GB | 0.20 GB | 66.0 GB |
| Q2_K | 46.5 GB | 0.20 GB | 52.3 GB |
| NVFP4cuda | 61.0 GB | 0.20 GB | 68.5 GB |
KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Qwen 3.5 122B-A10B (MoE) natively (33)
- NVIDIA B300 288GBBF16 · 77.8 t/s
- NVIDIA B200 180GBNVFP4 · 308.3 t/s
- NVIDIA H200 141GBQ6_K · 113.2 t/s
- NVIDIA H100 80GBQ3_K_M · 134.1 t/s
- NVIDIA A100 80GBQ3_K_M · 81.6 t/s
Show 28 more
- NVIDIA RTX Pro 6000NVFP4 · 51.8 t/s
- NVIDIA DGX Spark (128GB)NVFP4 · 10.5 t/s
- AMD Instinct MI300XQ8_0 · 96.7 t/s
- AMD Strix Halo (128GB)Q6_K · 6 t/s
- AMD Strix Halo (96GB)Q4_K_M · 8.1 t/s
- AMD Strix Halo (64GB)Q2_K · 12.9 t/s
- Apple M5 Ultra (512GB)BF16 · 14.4 t/s
- Apple M5 Ultra (256GB)Q8_0 · 26.9 t/s
- Apple M5 Ultra (96GB)Q4_K_M · 46.8 t/s
- Apple M5 Max (128GB)Q6_K · 17.8 t/s
- Apple M5 Max (64GB)Q2_K · 38.1 t/s
- Apple M5 Pro (64GB)Q2_K · 19 t/s
- Apple M4 Max (128GB)Q6_K · 15.8 t/s
- Apple M4 Max (64GB)Q2_K · 33.9 t/s
- Apple M3 Ultra (512GB)BF16 · 9.8 t/s
- Apple M3 Ultra (256GB)Q8_0 · 18.4 t/s
- Apple M3 Ultra (96GB)Q4_K_M · 32 t/s
- Apple M3 Max (128GB)Q6_K · 11.6 t/s
- Apple M3 Max (96GB)Q4_K_M · 11.7 t/s
- Apple M3 Max (64GB)Q2_K · 24.8 t/s
- Apple M2 Ultra (192GB)Q8_0 · 18 t/s
- Apple M2 Ultra (64GB)Q2_K · 49.6 t/s
- Apple M2 Max (96GB)Q4_K_M · 15.6 t/s
- Apple M2 Max (64GB)Q2_K · 24.8 t/s
- Apple M1 Ultra (128GB)Q6_K · 23.2 t/s
- Apple M1 Ultra (64GB)Q2_K · 49.6 t/s
- Apple M1 Max (64GB)Q2_K · 24.8 t/s
- Intel Data Center GPU Max 1550Q6_K · 77.2 t/s
Plus 11 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 5090Q2_K · 5.3 t/s
- NVIDIA A100 40GBQ2_K · 9.4 t/s
- NVIDIA L40SQ3_K_M · 5.9 t/s
- NVIDIA RTX A6000Q3_K_M · 5.8 t/s
- NVIDIA RTX 5000 AdaQ2_K · 4.9 t/s
- NVIDIA RTX 6000 AdaQ3_K_M · 6 t/s
- AMD Radeon PRO W7800Q2_K · 4.9 t/s
- AMD Radeon PRO W7900Q3_K_M · 5.9 t/s
- AMD Radeon AI PRO R9700 32GBQ2_K · 5 t/s
- Intel Arc Pro B70 32GBQ2_K · 5 t/s
- Intel Data Center GPU Max 1100Q3_K_M · 6.2 t/s
Notes
Hybrid dense-MoE stack: 48 layers built from twelve repeats of three Gated DeltaNet layers followed by one Gated Attention layer, so only 12 of the 48 layers keep a growing KV cache, the same architecture idea as Qwen 3.6/3.8 27B scaled up with MoE routing (256 experts, 8 routed plus 1 shared active). Context is 262,144 tokens natively and up to roughly 1,010,000 with extended settings.
Compare Qwen 3.5 122B-A10B (MoE) with other models
Continue reading
Frequently asked questions
- What are the VRAM requirements for Qwen 3.5 122B-A10B (MoE)?
- Qwen 3.5 122B-A10B (MoE) requires approximately 83.4 GB of VRAM at Q4_K_M quantization, 145.5 GB at Q8, and 273.5 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
- How many parameters does Qwen 3.5 122B-A10B (MoE) have?
- Qwen 3.5 122B-A10B (MoE) has 122 billion total parameters, but only 10 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is Qwen 3.5 122B-A10B (MoE)?
- Qwen 3.5 122B-A10B (MoE) achieves an MMLU-Pro score of 86.7, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
- Can Qwen 3.5 122B-A10B (MoE) run on a 16 GB GPU?
- No. At Q4_K_M, Qwen 3.5 122B-A10B (MoE) needs 83.4 GB of VRAM, more than 16 GB. You will need a multi-GPU server.
- Can Qwen 3.5 122B-A10B (MoE) run on a 24 GB GPU?
- No. Even at Q4_K_M, Qwen 3.5 122B-A10B (MoE) needs 83.4 GB. Consider a multi-GPU server with 84 GB+ of combined VRAM.
- What is the smallest quantization for Qwen 3.5 122B-A10B (MoE) that fits in 24 GB of VRAM?
- Qwen 3.5 122B-A10B (MoE) cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 52.3 GB at Q2_K.
- What GPU do I need to run Qwen 3.5 122B-A10B (MoE) locally?
- You need a multi-GPU server. At Q4_K_M, Qwen 3.5 122B-A10B (MoE) needs 83.4 GB VRAM, more than any single consumer GPU. That's roughly 2x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.