CanItRun Logocanitrun.

Qwen 3.5 35B-A3B (MoE)

Qwen 3.5 35B-A3B (MoE) needs roughly 24.1 GB VRAM at Q4_K_M quantization (78.6 GB at FP16). 84 GPUs we track can run it fully in VRAM at 8k context.

84 GPUs run this natively · 21 with CPU offload

Alibaba35B params3B active (MoE)256k contextApache 2.0Commercial use ok
Q4_K_M total
24.1 GB
at 8k context
Smallest GPU
32 GB
NVIDIA RTX 5090, at Q4_K_M
KV cache, full context
5.4 GB
10 of 40 layers cache
Inputs
Text · Image · Video
Apache 2.0, released February 2026
Active experts
8 routed + 1 shared
of 256 total
Linear attention layers
30 of 40
fixed-size state, no KV growth

Qwen 3.5 35B-A3B (MoE) is a Mixture of Experts (MoE) model with 35B total parameters but only 3B active per token developed by Alibaba. Released 24 February 2026 under Apache 2.0. Pairs MoE routing (256 experts, 8 routed plus 1 shared active) with the same hybrid attention idea Qwen used dense-side in Qwen 3.6 27B: 40 layers built from ten repeats of three Gated DeltaNet layers followed by one Gated Attention layer, so only 10 of 40 layers keep a growing KV cache. Context is 262,144 tokens natively and up to roughly 1M with YaRN scaling, and the checkpoint accepts text, image, and video input.

To run Qwen 3.5 35B-A3B (MoE) locally: VRAM is sized by the full 35B of weights, not the 3B active: Q4_K_M weights alone are roughly 21.3 GB, already close to what a 24 GB card can hold once overhead is added, so a 32 GB card is the more comfortable target. The hybrid stack keeps long context cheap: even the full 262,144-token window only adds about 5.4 GB of KV cache, versus roughly 21.5 GB if all 40 layers used full attention. As a MoE model, decode speed tracks the 3B active-parameter count once loaded, so tokens/sec sits much closer to a dense 3B model than a dense 35B one. As a MoE model, inference speed depends on active parameters (3B) rather than total size.

Alibaba reports MMLU-Pro 85.3, GPQA Diamond 84.2, and SWE-bench Verified 69.2, a large step up from Qwen3-30B-A3B on a model that still activates only 3B parameters per token. These are vendor-measured figures; independent third-party reproduction was still limited shortly after launch.

A mixture of experts built on a hybrid attention stack

Qwen 3.5 35B-A3B stacks two efficiency tricks instead of one. Like every A3B release, it's a mixture of experts: 256 experts live in the weights, but a router picks only 8 plus one shared expert per token, so only 3 of the 35 billion parameters actually do compute for that token. Underneath that routing, the 40 layers are not all the same kind of layer either: the same hybrid idea Qwen used dense-side in Qwen 3.6 27B, applied here to an MoE stack instead.

Gated DeltaNetlinear attention, fixed-size stateGated Attention2 KV heads × 256 dim
layer 13:1 pattern × 10layer 40

10 of 40 layers keep a KV cache that grows with the full context. The other 30 hold a fixed-size recurrent state, so their memory cost does not move as the context window grows.

Only the 10 Gated Attention layers keep a per-token KV cache; the 30 Gated DeltaNet layers carry a constant-size recurrent state that costs the same at 2,000 tokens or 200,000. Because the MoE routing already keeps active compute low, the hybrid attention layer stacks a second, independent saving on top: one shrinks the weights you read per token, the other shrinks the cache you read per token.

What a full 262,144-token context actually costs

A 35B-parameter model's KV cache would normally grow fast enough to dominate the VRAM total well before the advertised context ceiling. Caching only 10 of 40 layers instead of all of them cuts that bill to a quarter of what it would otherwise be.

0612182432k64k128k192k256k5.4 GBQwen 3.5 35B21.5 GBall-attn 35B
Qwen 3.5 35B-A3B (MoE): 10 of 40 layers cacheThe same 35B-A3B if all 40 layers used full attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Filling the whole 262,144-token window costs about 5.4 GB of KV cache instead of 21.5 GB. Add the 21.3 GB of Q4_K_M weights and the full-context total is 29.9 GB, comfortably inside a 32 GB card, even though the bare weights alone are already tight for a 24 GB one. The all-attention version of the same model would need 47.9 GB at the same context, past every consumer GPU this site tracks and into datacenter-card territory.

VRAM at each quantization

Calculated at 8k context. Because only part of this model's stack keeps a growing KV cache, longer sessions cost far less extra VRAM than a full-attention model of the same size.

QuantWeightsKV cacheTotal
FP32140.0 GB0.17 GB157.0 GB
BF1670.0 GB0.17 GB78.6 GB
FP1670.0 GB0.17 GB78.6 GB
Q8_037.2 GB0.17 GB41.9 GB
Q6_K28.7 GB0.17 GB32.4 GB
Q5_K_M24.9 GB0.17 GB28.1 GB
Q4_K_Mrec21.3 GB0.17 GB24.1 GB
Q3_K_M16.8 GB0.17 GB19.0 GB
Q2_K13.3 GB0.17 GB15.1 GB
NVFP4cuda17.5 GB0.17 GB19.8 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Quantization × context: where it fits

The table above holds context fixed at 8k. Add context as a second axis and the hybrid stack's story becomes visible: the total barely moves as context grows, because most of it is the fixed 35B weight footprint rather than a KV cache that scales with tokens.

Total VRAM needed by quantization and context length, with the GPU class each figure requires.
Quant8k ctx32k ctx128k ctx256k ctx
Q8_041.9 GB48 GB card42.4 GB48 GB card44.7 GB48 GB card47.7 GB48 GB card
Q6_K32.4 GB48 GB card32.9 GB48 GB card35.2 GB48 GB card38.2 GB48 GB card
Q5_K_M28.1 GB32 GB card28.7 GB32 GB card30.9 GB32 GB card33.9 GB48 GB card
Q4_K_Mrec24.1 GB32 GB card24.6 GB32 GB card26.9 GB32 GB card29.9 GB32 GB card
Q3_K_M19.0 GB24 GB card19.6 GB24 GB card21.9 GB24 GB card24.9 GB32 GB card
Each cell shows the smallest card that holds it:Consumer GPUWorkstation GPU

Benchmarks

MMLU-Pro
85.3
GPQA Diamond
84.2
SWE-bench Verified
69.2

GPUs that run Qwen 3.5 35B-A3B (MoE) natively (84)

Show 79 more
Plus 21 GPUs that run it with CPU offload (slower)

Notes

256 experts, 8 routed + 1 shared active per token. Hybrid Gated DeltaNet / Gated Attention stack keeps KV cache growth flat across most of the context window; vision needs the separate mmproj projector file alongside the weights.

Hugging Face ↗Ollama ↗Released 2026-02-24

Compare Qwen 3.5 35B-A3B (MoE) with other models

How to run Qwen 3.5 35B-A3B (MoE) locally

816244880160320

Q4_K_M needs 24.1 GB: needs a workstation or datacenter GPU (48–80 GB).

llama.cpp (thinking mode, Qwen's recommended sampling)

./llama-server \
  -hf unsloth/Qwen3.5-35B-A3B-GGUF:Q4_K_M \
  -c 32768 -ngl 99 \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0

Ollama

ollama run qwen3.5:35b-a3b

llama.cpp

# Non-thinking mode, for straight instruction following
./llama-server \
  -hf unsloth/Qwen3.5-35B-A3B-GGUF:Q4_K_M \
  -c 32768 -ngl 99 \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5 \
  --chat-template-kwargs '{"enable_thinking":false}'

# Vision needs the projector file alongside the weights
./llama-server -hf unsloth/Qwen3.5-35B-A3B-GGUF:Q4_K_M \
  --mmproj mmproj-F16.gguf

LM Studio: Search for 'Qwen3.5 35B A3B' and take the Q4_K_M build. Because most of the total size is MoE weights rather than KV cache, raising the context slider costs far less VRAM here than on a same-sized dense model; budget for the ~21 GB of weights first, then add context.

Why this quantization? Q4_K_M puts the 256-expert weights at roughly 21.3 GB regardless of context length, which is already close to what a 24 GB card can hold once activation overhead is added, so a 32 GB card is the comfortable target. The hybrid attention stack means context is cheap once you clear that weights floor: the KV cache itself only grows by about 5.2 GB going from 8k tokens to the full 262,144-token window, versus roughly 20.8 GB for an all-attention model of the same shape.

Who is Qwen 3.5 35B-A3B (MoE) for?

Anyone with a 32 GB GPU (or 32 GB+ unified-memory Apple Silicon) who wants near-dense-70B-class reasoning and coding scores at a fraction of the active compute, plus a long, cheap context window for feeding in large documents.

Best for

  • Long-document and multi-file analysis, where the hybrid attention stack keeps the KV cache from growing much
  • Agentic and tool-use workflows that benefit from a big context budget without a big VRAM budget
  • Image and video understanding via the vision projector
  • Users who want MoE-level throughput without a datacenter GPU

Not ideal for

  • 16-24 GB GPUs, which will offload part of the 35B weight footprint even at short context
  • Anyone who needs the 3B active-parameter count to translate into a small download, the full 35B still has to be stored
  • Latency-sensitive chat if thinking mode is left on, since it adds tokens before every answer
  • Workloads that need benchmark claims independently verified before adopting a model, third-party reproductions were still limited shortly after launch

So should you run it?

On a 24 GB card, Q4_K_M is close but doesn't quite clear the bar: the weights alone already use most of what a 24 GB card can hold, so expect a sliver of CPU offload even at short context, and more of one as context grows. A 32 GB card runs it fully in VRAM at every context up to the full 262,144-token window, which is the more comfortable target. Because it's a mixture of experts, decode speed tracks the 3B active-parameter count rather than the 35B total once the weights are loaded, so it should feel closer to a small dense model in responsiveness than its VRAM footprint suggests. The trade-off is the one every MoE model makes: you pay the full 35B in VRAM to get 3B-model speed, and the 256-expert routing means quality depends on exactly which experts a given prompt activates, something the aggregate benchmark scores don't fully capture.

Continue reading

vram-guides9 min
Best LLMs for 24 GB VRAM (2026)
vram-guides10 min
Qwen Family Guide: 2.5 to 3.6
hardware10 min
Best GPUs for Coding Agents in 2026

Frequently asked questions

What are the VRAM requirements for Qwen 3.5 35B-A3B (MoE)?
Qwen 3.5 35B-A3B (MoE) requires approximately 24.1 GB of VRAM at Q4_K_M quantization, 41.9 GB at Q8, and 78.6 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Qwen 3.5 35B-A3B (MoE) have?
Qwen 3.5 35B-A3B (MoE) has 35 billion total parameters, but only 3 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
How capable is Qwen 3.5 35B-A3B (MoE)?
Qwen 3.5 35B-A3B (MoE) achieves an MMLU-Pro score of 85.3, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
Can Qwen 3.5 35B-A3B (MoE) run on a 16 GB GPU?
No. At Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 24.1 GB of VRAM, more than 16 GB. You will need a 32 GB GPU like the RTX 5090.
Can Qwen 3.5 35B-A3B (MoE) run on a 24 GB GPU?
No. Even at Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 24.1 GB. Consider a 32 GB card like the RTX 5090.
What is the smallest quantization for Qwen 3.5 35B-A3B (MoE) that fits in 24 GB of VRAM?
At NVFP4, Qwen 3.5 35B-A3B (MoE) needs 19.8 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Qwen 3.5 35B-A3B (MoE) locally?
A 32 GB GPU is the minimum. At Q4_K_M, Qwen 3.5 35B-A3B (MoE) needs 24.1 GB VRAM. Good option: RTX 5090 (32 GB).