Qwen3 235B-A22B (MoE)
Qwen3 235B-A22B (MoE) needs roughly 117.5GB VRAM at Q4 quantization (470.0GB at FP16). 11 GPUs we track can run it fully in VRAM at 8k context.
Alibaba235B params22B active (MoE)128k contextApache 2.0Commercial use ok
VRAM at each quantization
Assumes 8k context. KV cache grows linearly with context length.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP16 | 470.0 GB | 1.58 GB | 528.2 GB |
| Q8 | 235.0 GB | 1.58 GB | 265.0 GB |
| Q6_K | 176.3 GB | 1.58 GB | 199.2 GB |
| Q5_K_M | 146.9 GB | 1.58 GB | 166.3 GB |
| Q4_K_M | 117.5 GB | 1.58 GB | 133.4 GB |
| Q3_K_M | 94.0 GB | 1.58 GB | 107.0 GB |
| Q2_K | 70.5 GB | 1.58 GB | 80.7 GB |
Benchmarks
GPUs that run Qwen3 235B-A22B (MoE) natively (11)
- AMD Instinct MI300XQ5_K_M · 424 t/s
- Apple M4 Ultra (384GB)Q8 · 54.6 t/s
- Apple M4 Ultra (192GB)Q5_K_M · 87.4 t/s
- Apple M4 Max (128GB)Q3_K_M · 68.3 t/s
- Apple M4 Max (96GB)Q2_K · 91 t/s
- Apple M3 Max (128GB)Q3_K_M · 50 t/s
- Apple M3 Max (96GB)Q2_K · 66.7 t/s
- Apple M2 Ultra (384GB)Q8 · 40 t/s
- Apple M2 Ultra (192GB)Q5_K_M · 64 t/s
- Apple M2 Max (96GB)Q2_K · 66.7 t/s
- Apple M1 Ultra (128GB)Q3_K_M · 100 t/s
Plus 2 GPUs that run it with CPU offload (slower)
- NVIDIA H100 80GBQ2_K · 126.9 t/s
- NVIDIA A100 80GBQ2_K · 77.2 t/s
Notes
Flagship reasoning MoE: 235B total, 22B active. Needs multi-GPU server.