CanItRun Logocanitrun.

Qwen 3.5 35B-A3B (MoE)

Qwen 3.5 35B-A3B (MoE) needs roughly 24.8 GB VRAM at Q4_K_M quantization (79.3 GB at FP16). 67 GPUs we track can run it fully in VRAM at 8k context.

67 GPUs run this natively · 27 with CPU offload

Alibaba35B params3B active (MoE)256k contextApache 2.0Commercial use ok

Qwen 3.5 35B-A3B (MoE) is a Mixture of Experts (MoE) model with 35B total parameters but only 3B active per token developed by Alibaba. February 2026 MoE release, Apache 2.0, with a 262K native context window and vision support. Continues the 30B-A3B-class efficiency pattern Qwen established with Qwen3-30B-A3B — 35B total parameters but only 3B active per token.

To run Qwen 3.5 35B-A3B (MoE) locally: VRAM is sized by the full 35B of weights, not the 3B active — Q4_K_M needs roughly 22GB, fitting a 24GB GPU. As a MoE model, throughput tracks the 3B active count once it's loaded, so tokens/sec is closer to a dense 3B model than a dense 35B one. As a MoE model, inference speed depends on active parameters (3B) rather than total size.

MMLU-Pro 84.2 is a large step up from Qwen3-30B-A3B, and the low active-parameter count keeps inference fast even though the total weight footprint governs VRAM.

VRAM at each quantization

Calculated at 8k context. Since KV cache scales linearly with context, longer sessions need more VRAM than shown here.

QuantWeightsKV cacheTotal
FP32140.0 GB0.81 GB157.7 GB
BF1670.0 GB0.81 GB79.3 GB
FP1670.0 GB0.81 GB79.3 GB
Q8_037.2 GB0.81 GB42.6 GB
Q6_K28.7 GB0.81 GB33.1 GB
Q5_K_M24.9 GB0.81 GB28.8 GB
Q4_K_Mrec21.3 GB0.81 GB24.8 GB
Q3_K_M16.8 GB0.81 GB19.8 GB
Q2_K13.3 GB0.81 GB15.8 GB
NVFP4cuda17.5 GB0.81 GB20.5 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Benchmarks

MMLU-Pro
84.2

GPUs that run Qwen 3.5 35B-A3B (MoE) natively (67)

Plus 27 GPUs that run it with CPU offload (slower)
Hugging Face ↗Ollama ↗Released 2026-02-15

Continue reading

vram-guides9 min
Best LLMs for 24 GB VRAM (2026)
vram-guides10 min
Qwen Family Guide: 2.5 to 3.6
hardware10 min
Best GPUs for Coding Agents in 2026

Frequently asked questions

What are the VRAM requirements for Qwen 3.5 35B-A3B (MoE)?
Qwen 3.5 35B-A3B (MoE) requires approximately 24.8 GB of VRAM at Q4_K_M quantization, 42.6 GB at Q8, and 79.3 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Qwen 3.5 35B-A3B (MoE) have?
Qwen 3.5 35B-A3B (MoE) has 35 billion total parameters, but only 3 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
How capable is Qwen 3.5 35B-A3B (MoE)?
Qwen 3.5 35B-A3B (MoE) achieves an MMLU-Pro score of 84.2, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
Can Qwen 3.5 35B-A3B (MoE) run on a 16 GB GPU?
No. At Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 24.8 GB of VRAM — more than 16 GB. You will need a 48 GB GPU like the RTX 6000 Ada or a dual-GPU setup.
Can Qwen 3.5 35B-A3B (MoE) run on a 24 GB GPU?
No. Even at Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 24.8 GB. Consider a 48 GB card like the RTX 6000 Ada or a dual RTX 4090 setup.
What is the smallest quantization for Qwen 3.5 35B-A3B (MoE) that fits in 24 GB of VRAM?
At NVFP4, Qwen 3.5 35B-A3B (MoE) needs 20.5 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Qwen 3.5 35B-A3B (MoE) locally?
You need a 48 GB GPU or a dual-GPU setup. At Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 24.8 GB VRAM. Options: RTX 6000 Ada (48 GB), A6000 (48 GB), or 2× RTX 4090.