DeepSeek V3 671B
DeepSeek V3 671B needs roughly 458.3 GB VRAM at Q4_K_M quantization (1503.6 GB at FP16). 2 GPUs we track can run it fully in VRAM at 8k context.
2 GPUs run this natively · 1 with CPU offload
DeepSeek V3 671B is a Mixture of Experts (MoE) model with 671B total parameters but only 37B active per token developed by DeepSeek. December 2024 671B MoE with Multi-Head Latent Attention (MLA), 16× smaller KV cache than standard GQA.
To run DeepSeek V3 671B locally: Full weights need roughly 408.6GB at this site's recommended Q4_K_M quantization, about 458.3GB total at a short context; no single machine this site tracks fits it, so a multi-GPU datacenter server is the realistic floor. MLA keeps the KV cache itself small, but doesn't change that weight-loading floor. As a MoE model, inference speed depends on active parameters (37B) rather than total size.
MMLU-Pro 75.9%, Math 90.2%, frontier-class quality with efficient attention.
VRAM at each quantization
DeepSeek V3 671B natively supports a longer context window, but the table below is capped at 8k for comparability; KV cache grows linearly with context length.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 2684.0 GB | 0.51 GB | 3006.7 GB |
| BF16 | 1342.0 GB | 0.51 GB | 1503.6 GB |
| FP16 | 1342.0 GB | 0.51 GB | 1503.6 GB |
| Q8_0 | 713.3 GB | 0.51 GB | 799.4 GB |
| Q6_K | 550.9 GB | 0.51 GB | 617.6 GB |
| Q5_K_M | 477.8 GB | 0.51 GB | 535.7 GB |
| Q4_K_Mrec | 408.6 GB | 0.51 GB | 458.3 GB |
| Q3_K_M | 322.8 GB | 0.51 GB | 362.1 GB |
| Q2_K | 255.7 GB | 0.51 GB | 286.9 GB |
| NVFP4cuda | 335.5 GB | 0.51 GB | 376.3 GB |
KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.
Benchmarks
GPUs that run DeepSeek V3 671B natively (2)
- Apple M5 Ultra (512GB)Q4_K_M · 12.7 t/s
- Apple M3 Ultra (512GB)Q4_K_M · 8.7 t/s
Plus 1 GPUs that run it with CPU offload (slower)
- NVIDIA B300 288GBQ2_K · 109.5 t/s
Notes
MoE with MLA attention: KV cache is ~16× smaller than standard GQA; kvHeads/headDim here approximates MLA storage.
Compare DeepSeek V3 671B with other models
How to run DeepSeek V3 671B locally
Q4_K_M needs 458.3 GB: needs multiple datacenter-class GPUs (80 GB+ each).
Ollama
ollama run deepseek-v3llama.cpp
./llama-cli -m deepseek-v3.Q4_K_M.gguf -c 8192 -ngl 99LM Studio: DeepSeek V3 is a 671B MoE model. At this site's recommended Q4_K_M it requires roughly 409 GB of weights alone. Check LM Studio for available GGUF splits if you have a multi-GPU setup.
Why this quantization? DeepSeek V3 has 671B total parameters in a Mixture-of-Experts architecture with only 37B active per token. The MoE design means all 671B parameters must be loaded into VRAM even though only a fraction is used per forward pass. Q4_K_M is the floor this site recommends, since quality loss below it is too severe to hold up as the default pick even at this size; that puts the weights at roughly 408.6 GB. The MLA (Multi-head Latent Attention) architecture keeps the KV cache exceptionally small, but doesn't reduce that weight floor.
Who is DeepSeek V3 671B for?
Researchers and organizations with multi-GPU servers (7+ high-VRAM GPUs) who want frontier-level open-weight performance. DeepSeek V3 matches or exceeds many commercial APIs on benchmarks, making it attractive for teams building self-hosted inference endpoints to replace paid API dependencies.
Best for
- Frontier-level general-purpose reasoning and analysis (MMLU-Pro: 75.9, GPQA: 59.1)
- Advanced mathematics and quantitative problem solving (MATH: 90.2)
- Code generation with competitive LiveCodeBench scores (40.5)
- Self-hosted API replacement for commercial LLM services
- Research evaluations requiring open-weight frontier models
Not ideal for
- Any setup with less than 7x 80 GB GPUs, at Q4_K_M the model needs roughly 409 GB for weights
- Interactive use cases where fast time-to-first-token matters -- MoE routing adds latency
- Budget-conscious deployments -- consider DeepSeek R1 Distill 32B for 95% of use cases
Continue reading
Frequently asked questions
- What are the VRAM requirements for DeepSeek V3 671B?
- DeepSeek V3 671B requires approximately 458.3 GB of VRAM at Q4_K_M quantization, 799.4 GB at Q8, and 1503.6 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does DeepSeek V3 671B have?
- DeepSeek V3 671B has 671 billion total parameters, but only 37 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is DeepSeek V3 671B?
- DeepSeek V3 671B achieves an MMLU-Pro score of 75.9, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
- Can DeepSeek V3 671B run on a 16 GB GPU?
- No. At Q4_K_M, DeepSeek V3 671B needs 458.3 GB of VRAM, more than 16 GB. You will need a multi-GPU server.
- Can DeepSeek V3 671B run on a 24 GB GPU?
- No. Even at Q4_K_M, DeepSeek V3 671B needs 458.3 GB. Consider a multi-GPU server with 459 GB+ of combined VRAM.
- What is the smallest quantization for DeepSeek V3 671B that fits in 24 GB of VRAM?
- DeepSeek V3 671B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 286.9 GB at Q2_K.
- What GPU do I need to run DeepSeek V3 671B locally?
- You need a multi-GPU server. At Q4_K_M, DeepSeek V3 671B needs 458.3 GB VRAM, more than any single consumer GPU. That's roughly 6x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.