Are distilled models MoE?

No — they are DENSE, based on Qwen2.5 and Llama 3 architectures. Only the full 671B is MoE. Standard dense VRAM formula applies.

Why does R1 generate so many tokens?

Chain-of-thought reasoning produces 12K-23K internal thinking tokens before the visible response. This makes R1 2-5x slower per query despite similar tok/s rates.

Can I run V4 locally?

Not yet. Weights unreleased as of July 2026. V4-Flash (284B MoE) may be more practical than current 671B models when available.

14B Q4 fits 12 GB, scores 69.7% AIME. 32B Q4 needs 24 GB, better reasoning at 30-42 tok/s. If you have 24 GB, go 32B.

DeepSeek Family Guide: R1, V3 & Distilled Models

CanItRunJuly 1, 202610 min readVRAM Guides

The 671B MoE Architecture

DeepSeek V3 (Dec 2024) and R1 (Jan 2025) share the same base: 671B total, 37B active, 256 experts (8 active). Multi-Head Latent Attention (MLA) reduces KV cache memory. Standard Q4_K_M requires ~404 GB (5x A100 80GB). Unsloth's 1.58-bit dynamic quant (IQ1_S) compresses to 131 GB — 2x H100 or Mac Ultra 192 GB. All 671B parameters must be loaded regardless of active count.

R1 vs V3: Reasoning Overhead

V3 is general-purpose with direct answers. R1 adds chain-of-thought via RL — generating internal reasoning tokens before responding. Critical: R1 produces 2-5x more tokens per query. A 5-second V3 query may take 15-25 seconds with R1. V3-0324 (March 2025) incorporated R1's techniques for improved quality without the thinking overhead.

Distilled Models: Dense, Not MoE

CRITICAL: Distilled models are DENSE. They use Qwen2.5/Llama 3 architectures. 14B Qwen distill at Q4 (~9 GB) scores 69.7% AIME — near GPT-4o reasoning on a $200 GPU. 32B Qwen distill at Q4 (~20 GB, 30-42 tok/s on RTX 4090) is the best single-GPU option. 70B Llama distill at Q4 (~40-43 GB) needs dual GPUs. Only 8B and 671B got the R1-0528 update; other distills are January 2025 weights.

Full 671B on Consumer Hardware?

Not practical. 4x RTX 4090 + 384 GB RAM achieves 7-8 tok/s short context, degrading to 1-2 tok/s at long context. GPU utilization only 1-3% — CPU/RAM is the bottleneck. Mac Ultra 192 GB: 2-3 tok/s at 1.58-bit. Consensus: run the 32B distill on a 24 GB GPU — 90% of the reasoning at 1% of the hardware cost.

V4 (April 2026)

V4-Pro (1.6T/49B active) and V4-Flash (284B/13B active) with hybrid compressed attention. At 1M context, V4-Pro uses 27% of FLOPs and 10% of KV cache vs V3.2. Weights not yet publicly available but represent a major efficiency leap.

Which DeepSeek for Your GPU?

8-12 GB: 7B or 14B distill at Q4. 16 GB: 14B distill at Q8. 24 GB: 32B distill at Q4 (~20 GB, 30-42 tok/s) — best balance. 48 GB: 70B distill Q4 (~40-43 GB) on dual 3090s. 80 GB+: full R1 at Q4 (404 GB) or IQ1_S (131 GB). For 99% of users, the distilled models are the practical choice.

Frequently asked questions

Are distilled models MoE?: No — they are DENSE, based on Qwen2.5 and Llama 3 architectures. Only the full 671B is MoE. Standard dense VRAM formula applies.
Why does R1 generate so many tokens?: Chain-of-thought reasoning produces 12K-23K internal thinking tokens before the visible response. This makes R1 2-5x slower per query despite similar tok/s rates.
Can I run V4 locally?: Not yet. Weights unreleased as of July 2026. V4-Flash (284B MoE) may be more practical than current 671B models when available.
14B vs 32B distill?: 14B Q4 fits 12 GB, scores 69.7% AIME. 32B Q4 needs 24 GB, better reasoning at 30-42 tok/s. If you have 24 GB, go 32B.