CanItRun Logocanitrun.

DeepSeek R1 Distill Llama 8B

DeepSeek R1 Distill Llama 8B needs roughly 6.7 GB VRAM at Q4_K_M quantization (19.1 GB at FP16). 102 GPUs we track can run it fully in VRAM at 8k context.

102 GPUs run this natively · 2 with CPU offload

DeepSeek8B params125k contextMITCommercial use ok

DeepSeek R1 Distill Llama 8B is a 8B parameter dense model developed by DeepSeek. Compact 8B distillation bringing R1 reasoning to edge devices.

To run DeepSeek R1 Distill Llama 8B locally: Q5_K_M ~6GB — runs on 8GB GPUs. Best reasoning model for budget hardware.

MMLU-Pro 41.0%, GPQA 49.0%, Math 89.1% — exceptional reasoning for the size.

VRAM at each quantization

Numbers here are computed at 8k context. Because KV cache grows linearly with context length, expect higher totals at longer sequence lengths.

QuantWeightsKV cacheTotal
FP3232.0 GB1.07 GB37.0 GB
BF1616.0 GB1.07 GB19.1 GB
FP1616.0 GB1.07 GB19.1 GB
Q8_08.5 GB1.07 GB10.7 GB
Q6_K6.6 GB1.07 GB8.6 GB
Q5_K_Mrec5.7 GB1.07 GB7.6 GB
Q4_K_M4.9 GB1.07 GB6.7 GB
Q3_K_M3.9 GB1.07 GB5.5 GB
Q2_K3.0 GB1.07 GB4.6 GB
NVFP4cuda4.0 GB1.07 GB5.7 GB

Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.

Benchmarks

MMLU-Pro
41
GPQA Diamond
49
MATH
89.1

GPUs that run DeepSeek R1 Distill Llama 8B natively (102)

Plus 2 GPUs that run it with CPU offload (slower)
Hugging Face ↗Ollama ↗Released 2025-01-20

Compare DeepSeek R1 Distill Llama 8B with other models

How to run DeepSeek R1 Distill Llama 8B locally

816244880160320

Q5_K_M needs 7.6 GB — fits a single high-end consumer GPU (24 GB).

Ollama

ollama run deepseek-r1:8b

llama.cpp

./llama-cli -m deepseek-r1-distill-llama-8b.Q5_K_M.gguf -c 8192 -ngl 99

LM Studio: Search for 'DeepSeek R1 Distill Llama 8B' in LM Studio. The Q5_K_M variant runs on any 8 GB GPU -- the most accessible reasoning model available.

Why this quantization? At 8B dense parameters, Q5_K_M costs only about 5 GB of VRAM, easily fitting on budget GPUs. Despite its small size, this distill achieves an astonishing MATH score of 89.1 and GPQA of 49.0 -- better than many models five times its size. Q5 preserves these capabilities better than Q4, and the extra 700 MB is negligible on modern hardware.

Who is DeepSeek R1 Distill Llama 8B for?

Students, hobbyists, and developers with budget GPUs (8 GB VRAM) who want reasoning capabilities on accessible hardware. This is the easiest way to experience chain-of-thought reasoning locally. The MIT license and tiny footprint make it ideal for embedded applications or running alongside other workloads.

Best for

  • Math homework help and STEM tutoring at remarkable quality for the size
  • Affordable chain-of-thought reasoning on consumer hardware
  • Learning about reasoning models without expensive GPU requirements
  • Lightweight reasoning assistant that runs alongside your IDE or browser

Not ideal for

  • General knowledge queries (MMLU-Pro: 41.0 is limited for the model's size)
  • Complex instruction following where larger models are needed
  • Professional applications where accuracy needs to be consistently high

Continue reading

vram-guides10 min
DeepSeek Family Guide: R1, V3 & Distilled Models

Frequently asked questions

What are the VRAM requirements for DeepSeek R1 Distill Llama 8B?
DeepSeek R1 Distill Llama 8B requires approximately 6.7 GB of VRAM at Q4_K_M quantization, 10.7 GB at Q8, and 19.1 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does DeepSeek R1 Distill Llama 8B have?
DeepSeek R1 Distill Llama 8B has 8 billion parameters.
Is DeepSeek R1 Distill Llama 8B good at reasoning and math?
Yes. With a MATH score of 89.1 and MMLU-Pro of 41, DeepSeek R1 Distill Llama 8B handles complex multi-step reasoning, analytical tasks, and problem-solving well.
Can DeepSeek R1 Distill Llama 8B run on a 16 GB GPU?
Yes. DeepSeek R1 Distill Llama 8B needs 6.7 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 4070 Ti Super.
What is the smallest quantization for DeepSeek R1 Distill Llama 8B that fits in 24 GB of VRAM?
At BF16, DeepSeek R1 Distill Llama 8B needs 19.1 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run DeepSeek R1 Distill Llama 8B locally?
A 16 GB GPU is enough. At Q4_K_M, DeepSeek R1 Distill Llama 8B needs 6.7 GB VRAM. Good options: RTX 4080 (16 GB), RTX 4070 Ti Super (16 GB).