Qwen 3.6 35B
Qwen 3.6 35B needs roughly 22.0GB VRAM at Q4_K_M quantization (80.8GB at FP16). 49 GPUs we track can run it fully in VRAM at 8k context.
Alibaba35B params256k contextApache 2.0Commercial use ok
VRAM at each quantization
Assumes 8k context. KV cache grows linearly with context length.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP16 | 70.0 GB | 2.15 GB | 80.8 GB |
| Q8 | 35.0 GB | 2.15 GB | 41.6 GB |
| Q6_K | 26.3 GB | 2.15 GB | 31.8 GB |
| Q5_K_M | 21.9 GB | 2.15 GB | 26.9 GB |
| Q4_K_M | 17.5 GB | 2.15 GB | 22.0 GB |
| Q3_K_M | 14.0 GB | 2.15 GB | 18.1 GB |
| Q2_K | 10.5 GB | 2.15 GB | 14.2 GB |
Benchmarks
Benchmarks for this model are not yet available on the Open LLM Leaderboard v2. This is common for recently released models. Check back soon.
GPUs that run Qwen 3.6 35B natively (49)
- NVIDIA RTX 5090Q5_K_M · 81.9 t/s
- NVIDIA RTX 4090Q4_K_M · 57.6 t/s
- NVIDIA RTX 4080Q2_K · 68.3 t/s
- NVIDIA RTX 4060 Ti 16GBQ2_K · 27.4 t/s
- NVIDIA RTX 3090Q4_K_M · 53.5 t/s
- NVIDIA RTX 3090 TiQ4_K_M · 57.6 t/s
- NVIDIA H100 80GBQ8 · 95.7 t/s
- NVIDIA A100 80GBQ8 · 58.3 t/s
- NVIDIA A100 40GBQ6_K · 59.2 t/s
- NVIDIA L40SQ8 · 24.7 t/s
- NVIDIA RTX A6000Q8 · 21.9 t/s
- NVIDIA RTX 6000 AdaQ8 · 27.4 t/s
- NVIDIA DGX Spark (128GB)FP16 · 3.9 t/s
- AMD Radeon RX 7900 XTXQ4_K_M · 54.9 t/s
- AMD Radeon RX 7900 XTQ3_K_M · 57.1 t/s
- AMD Radeon RX 6800 XTQ2_K · 48.8 t/s
- AMD Instinct MI300XFP16 · 75.7 t/s
- AMD Strix Halo (128GB)FP16 · 3.7 t/s
- AMD Strix Halo (96GB)FP16 · 3.7 t/s
- AMD Strix Halo (64GB)Q8 · 7.3 t/s
- Apple M4 Ultra (384GB)FP16 · 15.6 t/s
- Apple M4 Ultra (192GB)FP16 · 15.6 t/s
- Apple M4 Max (128GB)FP16 · 7.8 t/s
- Apple M4 Max (96GB)FP16 · 7.8 t/s
- Apple M4 Max (64GB)Q8 · 15.6 t/s
- Apple M4 Max (48GB)Q8 · 15.6 t/s
- Apple M4 Pro (48GB)Q8 · 7.8 t/s
- Apple M4 Pro (24GB)Q3_K_M · 19.5 t/s
- Apple M4 (32GB)Q5_K_M · 5.5 t/s
- Apple M3 Max (128GB)FP16 · 5.7 t/s
- Apple M3 Max (96GB)FP16 · 5.7 t/s
- Apple M3 Max (64GB)Q8 · 11.4 t/s
- Apple M3 Max (48GB)Q8 · 11.4 t/s
- Apple M3 Max (36GB)Q6_K · 15.2 t/s
- Apple M3 Pro (36GB)Q6_K · 5.7 t/s
- Apple M3 (24GB)Q3_K_M · 7.1 t/s
- Apple M2 Ultra (384GB)FP16 · 11.4 t/s
- Apple M2 Ultra (192GB)FP16 · 11.4 t/s
- Apple M2 Max (96GB)FP16 · 5.7 t/s
- Apple M2 Max (64GB)Q8 · 11.4 t/s
- Apple M2 Max (32GB)Q5_K_M · 18.3 t/s
- Apple M2 Pro (32GB)Q5_K_M · 9.1 t/s
- Apple M2 (24GB)Q3_K_M · 7.1 t/s
- Apple M1 Ultra (128GB)FP16 · 11.4 t/s
- Apple M1 Ultra (64GB)Q8 · 22.9 t/s
- Apple M1 Max (64GB)Q8 · 11.4 t/s
- Apple M1 Max (32GB)Q5_K_M · 18.3 t/s
- Apple M1 Pro (32GB)Q5_K_M · 9.1 t/s
- Intel Arc A770 16GBQ2_K · 53.3 t/s
Plus 6 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 4070 TiQ6_K · 4.8 t/s
- NVIDIA RTX 4070Q6_K · 4.8 t/s
- NVIDIA RTX 4060Q6_K · 2.6 t/s
- NVIDIA RTX 3080 10GBQ6_K · 7.2 t/s
- NVIDIA RTX 3060 12GBQ6_K · 3.4 t/s
- CPU only (system RAM)Q5_K_M · 0.5 t/s
Notes
Larger dense sibling of Qwen3.6-27B; same reasoning and agentic capabilities.