DeepSeek V3 671B
DeepSeek V3 671B needs roughly 335.5GB VRAM at Q4 quantization (1342.0GB at FP16). 2 GPUs we track can run it fully in VRAM at 8k context.
DeepSeek671B params37B active (MoE)125k contextMITCommercial use ok
VRAM at each quantization
Assumes 8k context. KV cache grows linearly with context length.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP16 | 1342.0 GB | 0.51 GB | 1503.6 GB |
| Q8 | 671.0 GB | 0.51 GB | 752.1 GB |
| Q6_K | 503.3 GB | 0.51 GB | 564.2 GB |
| Q5_K_M | 419.4 GB | 0.51 GB | 470.3 GB |
| Q4_K_M | 335.5 GB | 0.51 GB | 376.3 GB |
| Q3_K_M | 268.4 GB | 0.51 GB | 301.2 GB |
| Q2_K | 201.3 GB | 0.51 GB | 226.0 GB |
Benchmarks
GPUs that run DeepSeek V3 671B natively (2)
- Apple M4 Ultra (384GB)Q4_K_M · 64.9 t/s
- Apple M2 Ultra (384GB)Q4_K_M · 47.6 t/s
Notes
MoE with MLA attention — KV cache is ~16× smaller than standard GQA; kvHeads/headDim here approximates MLA storage.