Qwen Family Guide: 2.5 to 3.6
Four Generations in Two Years
Alibaba's Qwen family has become the dominant force in local LLMs. Each generation's model performs roughly one tier above the previous: Qwen 3 8B matches Qwen 2.5 14B, Qwen 3.5 9B beats Qwen 3 14B. Qwen 2.5 established the baseline (0.5B-72B dense, Coder 32B at 92.7% HumanEval). Qwen 3 introduced MoE and dual-mode thinking. Qwen 3.5 brought Gated DeltaNet (~40% less KV cache). Qwen 3.6 (April 2026) focused on quality with the 27B dense (SWE-bench 77.2, ties Sonnet 4.5) and 35B-A3B MoE. Here is the real Q4_K_M weight size across a representative slice of the family, dense and MoE side by side:
Gated DeltaNet and the 3.5 Revolution
Qwen 3.5 (Feb-Mar 2026) introduced Gated DeltaNet: 3 DeltaNet layers per 1 full attention layer, reducing KV cache memory by ~40%. The 35B-A3B MoE achieves 112 tok/s on RTX 3090 and 207 tok/s on RTX 4090, near-32B-dense quality at small-model speeds. The 9B dense at Q4 (~5.5 GB) is the best model for 8 GB GPUs. Eight model sizes from 0.8B to 397B-A17B MoE. Ollama 0.17.4+ required.
Qwen 3.6: The 24 GB Champion
Qwen 3.6-27B at Q4 (~17 GB) scores 77.2 SWE-bench (ties Claude Sonnet 4.5), 94.1 AIME26, 86.2 MMLU-Pro. This is the best dense open-source model you can run on a single 24 GB GPU in mid-2026. The 35B-A3B MoE at Q4 (~20 GB) scores 73.4 SWE-bench with 86.0 GPQA Diamond. Both use the Gated DeltaNet architecture with 256K native context.
Coder Models
Qwen 2.5 Coder 32B (92.7% HumanEval) set the standard. Coder-Next (80B-A3B, 512 experts, Feb 2026): SWE-bench 70.6% with only 3B active, runs at 60-70 tok/s on consumer hardware. However, Qwen 3.6-27B at 77.2 SWE-bench outperforms all dedicated Coder variants while being smaller. For coding on a 24 GB GPU, Qwen 3.6-27B is the best choice.
Which Qwen for Your GPU?
8 GB: Qwen 3.5 9B Q4 (~5.5 GB). 12-16 GB: Qwen 3 14B Q4 (~9 GB) or Qwen 3.5 35B-A3B MoE with CPU offloading. 24 GB: Qwen 3.6-27B Q4 (~17 GB), the best thing on a single consumer GPU. 48 GB: Qwen 3.6-27B Q8_0 (~29 GB) for near-lossless quality with room for 128K+ context, or Qwen 3.6-35B Q8_0 (~37 GB) if you want the larger dense model, both beat Qwen 2.5 72B Q4 (~44 GB), which is no longer the best use of 48 GB now that a newer, smaller Qwen release exists at near-full precision. The Qwen family offers a model for every VRAM tier with Apache 2.0 licensing.
Frequently asked questions
- Which Qwen for 24 GB?
- Qwen 3.6-27B at Q4 (~17 GB, SWE-bench 77.2). For MoE speed, Qwen 3.5 35B-A3B at Q4 (~20 GB, 112 tok/s on RTX 3090).
- What is Gated DeltaNet?
- A hybrid architecture replacing some attention layers, reducing KV cache by ~40%. Enables longer context on the same hardware. Requires Ollama 0.17.4+.
- Are Qwen models open source?
- Yes, Apache 2.0 for all generations. Full commercial use, modification, and redistribution allowed.
- Qwen 3.5 vs 3.6?
- 3.5 offers more sizes (8 models). 3.6 offers better quality at 27B and 35B-A3B. If a 3.6 variant exists at your target size, prefer it.