Are distilled models MoE?

No, they are DENSE, based on Qwen2.5 and Llama 3 architectures. Only the full 671B and V4 models are MoE. Standard dense VRAM formula applies to the distills.

Why does R1 generate so many tokens?

Chain-of-thought reasoning produces 12K-23K internal thinking tokens before the visible response. This makes R1 2-5x slower per query despite similar tok/s rates.

Can I run V4 locally?

Yes, as of April 24, 2026: both V4-Pro and V4-Flash shipped as public, MIT-licensed weights on Hugging Face, with GGUF quants already available. Neither fits consumer VRAM tiers: V4-Flash (284B total, 13B active) needs roughly 108 GB even at Q2_K, and V4-Pro (1.6T total) needs genuine multi-node datacenter hardware. V4-Flash is meaningfully more accessible than full R1/V3 671B, though, needing under half the VRAM at the same quant level.

14B Q4 fits 12 GB, scores 69.7% AIME. 32B Q4 needs 24 GB, better reasoning at 30-42 tok/s. If you have 24 GB, go 32B.

DeepSeek Family Guide: R1, V3 & Distilled Models

CanItRunJuly 30, 202610 min readVRAM Guides

The 671B MoE Architecture

DeepSeek V3 (Dec 2024) and R1 (Jan 2025) share the same base: 671B total, 37B active, 256 experts (8 active). Multi-Head Latent Attention (MLA) reduces KV cache memory. Standard Q4_K_M requires ~404 GB (5x A100 80GB). Unsloth's 1.58-bit dynamic quant (IQ1_S) compresses to 131 GB: 2x H100 or Mac Ultra 192 GB. All 671B parameters must be loaded regardless of active count.

R1 vs V3: Reasoning Overhead

V3 is general-purpose with direct answers. R1 adds chain-of-thought via RL: generating internal reasoning tokens before responding. Critical: R1 produces 2-5x more tokens per query. A 5-second V3 query may take 15-25 seconds with R1. V3-0324 (March 2025) incorporated R1's techniques for improved quality without the thinking overhead.

Distilled Models: Dense, Not MoE

CRITICAL: Distilled models are DENSE. They use Qwen2.5/Llama 3 architectures. The 8B Llama distill at Q4 (~4.9 GB) is the lightest reasoning-capable option. 32B Qwen distill at Q4 (~19.8 GB, 30-42 tok/s on RTX 4090) is the best single-GPU option. 70B Llama distill at Q4 (~42.6 GB weights, ~49 GB total) needs three GPUs to sit fully in VRAM, not two. Only 8B and 671B got the R1-0528 update; other distills are January 2025 weights. Full R1 dwarfs every distill on this chart even though only 37B of its 671B parameters are active per token; MoE saves compute, not the VRAM you need to load it:

DeepSeek R1 Distill Llama 8B

4.9 GB

DeepSeek R1 Distill Qwen 32B

19.8 GB

DeepSeek R1 Distill Llama 70B

42.6 GB

DeepSeek R1 671B

408.6 GB

Full 671B on Consumer Hardware?

Not practical. 4x RTX 4090 + 384 GB RAM achieves 7-8 tok/s short context, degrading to 1-2 tok/s at long context. GPU utilization only 1-3%: CPU/RAM is the bottleneck. Mac Ultra 192 GB: 2-3 tok/s at 1.58-bit. Consensus: run the 32B distill on a 24 GB GPU: 90% of the reasoning at 1% of the hardware cost.

V4 (April 2026): Now Actually Available

V4-Pro (1.6T total, 49B active) and V4-Flash (284B total, 13B active) shipped April 24, 2026 with hybrid CSA/HCA compressed attention, both MIT-licensed with weights publicly available on Hugging Face, including community GGUF quantizations. At 1M context, V4-Pro uses 27% of the FLOPs and 10% of the KV cache that V3.2 needed for the same workload, a major efficiency leap. Neither is a consumer-VRAM model: V4-Pro at its recommended Q2_K still needs multi-GPU datacenter hardware. V4-Flash is the more approachable of the two: at Q2_K it needs roughly 108 GB, well under half of what full R1/V3 671B need at the same quant level, putting it within reach of a 2-3x A100/H100 80GB setup or a high-memory Mac Studio rather than requiring the 4-5 GPU setups full R1 needs. It's still not a single-consumer-GPU model, but it is the most practical way to get V4-generation quality today.

Which DeepSeek for Your GPU?

8-12 GB: 7B or 14B distill at Q4. 16 GB: 14B distill at Q8. 24 GB: 32B distill at Q4 (~19.8 GB, 30-42 tok/s), best balance. 48 GB and up: 70B distill Q4 (~42.6 GB weights, ~49 GB total, needs three GPUs pooled, not two, to avoid CPU offload). 80 GB+: V4-Flash at Q2_K (~108 GB) for V4-generation quality, full R1 at Q4 (~409 GB) or IQ1_S (131 GB) for the previous generation, or V4-Pro (1.6T) only on genuine multi-node datacenter hardware. For 99% of users, the distilled models are still the practical choice.

Frequently asked questions

Are distilled models MoE?: No, they are DENSE, based on Qwen2.5 and Llama 3 architectures. Only the full 671B and V4 models are MoE. Standard dense VRAM formula applies to the distills.
Why does R1 generate so many tokens?: Chain-of-thought reasoning produces 12K-23K internal thinking tokens before the visible response. This makes R1 2-5x slower per query despite similar tok/s rates.
Can I run V4 locally?: Yes, as of April 24, 2026: both V4-Pro and V4-Flash shipped as public, MIT-licensed weights on Hugging Face, with GGUF quants already available. Neither fits consumer VRAM tiers: V4-Flash (284B total, 13B active) needs roughly 108 GB even at Q2_K, and V4-Pro (1.6T total) needs genuine multi-node datacenter hardware. V4-Flash is meaningfully more accessible than full R1/V3 671B, though, needing under half the VRAM at the same quant level.
14B vs 32B distill?: 14B Q4 fits 12 GB, scores 69.7% AIME. 32B Q4 needs 24 GB, better reasoning at 30-42 tok/s. If you have 24 GB, go 32B.