CanItRun Logocanitrun.

Qwen 3.5 35B-A3B (MoE)

Qwen 3.5 35B-A3B (MoE) needs roughly 23.0 GB VRAM at Q4_K_M quantization (79.3 GB at FP16). 76 GPUs we track can run it fully in VRAM at 8k context.

76 GPUs run this natively · 19 with CPU offload

Alibaba35B params3B active (MoE)256k contextApache 2.0Commercial use ok

Qwen 3.5 35B-A3B (MoE) is a Mixture of Experts (MoE) large language model with 35B total parameters but only 3B active per token developed by Alibaba. Released in February 2026, it supports text and vision inputs with a 128K context window, released under the Apache 2.0 license, allowing commercial use.

To run Qwen 3.5 35B-A3B (MoE) locally, you need approximately 23.0 GB of VRAM at Q4_K_M quantization with 8k context. 76 of the GPUs we track can run it fully in VRAM, with a further 19 able to offload to system RAM. At Q4_K_M it requires 23.0 GB — a 24 GB GPU like the RTX 4090 is the sweet spot. At Q8_K_M (40.1 GB), you get near-FP16 quality while still fitting on 48 and 80 GB GPUs. FP16 requires 79.3 GB, limiting it to 48 and 80 GB GPUs. As a MoE model, inference speed depends on active parameters (3B) rather than total size, so once it fits in VRAM it runs noticeably faster than dense models of the same parameter count.

Its MMLU-Pro score of 84.2 places it among the strongest open-weight models available. The license allows commercial use.

VRAM at each quantization

Assumes 8k context. KV cache grows linearly with context length.

QuantWeightsKV cacheTotal
FP32140.0 GB0.81 GB157.7 GB
BF1670.0 GB0.81 GB79.3 GB
FP1670.0 GB0.81 GB79.3 GB
Q8_035.0 GB0.81 GB40.1 GB
Q6_K28.7 GB0.81 GB33.0 GB
Q5_K_M22.5 GB0.81 GB26.1 GB
Q4_K_Mrec19.7 GB0.81 GB23.0 GB
Q3_K_M15.1 GB0.81 GB17.8 GB
Q2_K11.5 GB0.81 GB13.8 GB
NVFP4cuda17.5 GB0.81 GB20.5 GB

KV cache shown at 8k context (FP16). NVFP4 requires a CUDA GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.

Benchmarks

MMLU-Pro
84.2

GPUs that run Qwen 3.5 35B-A3B (MoE) natively (76)

Plus 19 GPUs that run it with CPU offload (slower)
Hugging Face ↗Ollama ↗Released 2026-02-15

Frequently asked questions

What are the VRAM requirements for Qwen 3.5 35B-A3B (MoE)?
Qwen 3.5 35B-A3B (MoE) requires approximately 23.0 GB of VRAM at Q4_K_M quantization, 40.1 GB at Q8, and 79.3 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Qwen 3.5 35B-A3B (MoE) have?
Qwen 3.5 35B-A3B (MoE) has 35 billion total parameters, but only 3 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
How capable is Qwen 3.5 35B-A3B (MoE)?
Qwen 3.5 35B-A3B (MoE) achieves an MMLU-Pro score of 84.2, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
Can Qwen 3.5 35B-A3B (MoE) run on a 16 GB GPU?
No. At Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 23.0 GB of VRAM — more than 16 GB. You will need a 24 GB GPU like the RTX 4090 or RTX 3090.
Can Qwen 3.5 35B-A3B (MoE) run on a 24 GB GPU?
Yes. Qwen 3.5 35B-A3B (MoE) fits in a 24 GB GPU at Q4_K_M, requiring 23.0 GB VRAM. GPUs with 24 GB include the RTX 4090, RTX 3090, and RTX 3090 Ti.
What is the smallest quantization for Qwen 3.5 35B-A3B (MoE) that fits in 24 GB of VRAM?
At NVFP4, Qwen 3.5 35B-A3B (MoE) needs 20.5 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Qwen 3.5 35B-A3B (MoE) locally?
A 24 GB GPU is the minimum. At Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 23.0 GB VRAM. Good options: RTX 4090 (24 GB), RTX 3090 (24 GB).