CanItRun Logocanitrun.

Qwen 3.5 9B

Qwen 3.5 9B needs roughly 6.4 GB VRAM at Q4_K_M quantization (20.5 GB at FP16). 101 GPUs we track can run it fully in VRAM at 8k context.

101 GPUs run this natively · 2 with CPU offload

Alibaba9B params256k contextApache 2.0Commercial use ok

Qwen 3.5 9B is a 9B parameter dense large language model developed by Alibaba. Released in February 2026, it supports text and vision inputs with a 256K context window, released under the Apache 2.0 license, allowing commercial use. Thinking mode on by default. Hybrid Gated DeltaNet / Gated Attention stack keeps KV cache growth flat across most of the context window; vision needs the separate mmproj projector file alongside the weights.

To run Qwen 3.5 9B locally, you need approximately 6.4 GB of VRAM at Q4_K_M quantization with 8k context. 101 of the GPUs we track can run it fully in VRAM, with a further 2 able to offload to system RAM. At Q4_K_M it needs just 6.4 GB, making it accessible even on mid-range 16 GB cards like RTX 4080 and RTX 5070 Ti. At Q8_K_M (11.0 GB), you get near-FP16 quality while still fitting on 12, 16, 24, 32, 48 and 80 GB GPUs. FP16 requires 20.5 GB, limiting it to 48 and 80 GB GPUs.

Its MMLU-Pro score of 82.5 places it among the strongest open-weight models available. The license allows commercial use.

VRAM at each quantization

Calculated at 8k context. Because only part of this model's stack keeps a growing KV cache, longer sessions cost far less extra VRAM than a full-attention model of the same size.

QuantWeightsKV cacheTotal
FP3236.0 GB0.27 GB40.6 GB
BF1618.0 GB0.27 GB20.5 GB
FP1618.0 GB0.27 GB20.5 GB
Q8_09.6 GB0.27 GB11.0 GB
Q6_K7.4 GB0.27 GB8.6 GB
Q5_K_M6.4 GB0.27 GB7.5 GB
Q4_K_Mrec5.5 GB0.27 GB6.4 GB
Q3_K_M4.3 GB0.27 GB5.2 GB
Q2_K3.4 GB0.27 GB4.1 GB
NVFP4cuda4.5 GB0.27 GB5.3 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Benchmarks

MMLU-Pro
82.5
GPQA Diamond
81.7
IFEval
91.5

GPUs that run Qwen 3.5 9B natively (101)

Show 96 more
Plus 2 GPUs that run it with CPU offload (slower)

Notes

Thinking mode on by default. Hybrid Gated DeltaNet / Gated Attention stack keeps KV cache growth flat across most of the context window; vision needs the separate mmproj projector file alongside the weights.

Hugging Face ↗Ollama ↗Released 2026-02-24

Continue reading

vram-guides8 min
Best LLMs You Can Run on 8 GB VRAM (2026)
vram-guides8 min
Best LLMs for 6 GB VRAM (2026)
vram-guides8 min
Best LLMs for 12 GB VRAM (2026)

Frequently asked questions

What are the VRAM requirements for Qwen 3.5 9B?
Qwen 3.5 9B requires approximately 6.4 GB of VRAM at Q4_K_M quantization, 11.0 GB at Q8, and 20.5 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Qwen 3.5 9B have?
Qwen 3.5 9B has 9 billion parameters.
How capable is Qwen 3.5 9B?
Qwen 3.5 9B achieves an MMLU-Pro score of 82.5, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
Can Qwen 3.5 9B run on a 16 GB GPU?
Yes. Qwen 3.5 9B needs 6.4 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 5070 Ti.
What is the smallest quantization for Qwen 3.5 9B that fits in 24 GB of VRAM?
At BF16, Qwen 3.5 9B needs 20.5 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Qwen 3.5 9B locally?
A 16 GB GPU is enough. At Q4_K_M, Qwen 3.5 9B needs 6.4 GB VRAM. Good options: RTX 4080 (16 GB), RTX 5070 Ti (16 GB).