CanItRun Logocanitrun.

Qwen3 8B

Qwen3 8B needs roughly 6.8 GB VRAM at Q4_K_M quantization (19.3 GB at FP16). 102 GPUs we track can run it fully in VRAM at 8k context.

102 GPUs run this natively · 2 with CPU offload

Alibaba8B params128k contextApache 2.0Commercial use ok

Qwen3 8B is a 8B parameter dense model developed by Alibaba. April 2025 dense 8B model in the Qwen3 family, Apache 2.0 licensed, sharing the same thinking/non-thinking hybrid design as the rest of the lineup.

To run Qwen3 8B locally: Q5_K_M needs roughly 5-6GB — runs on any 8GB+ GPU. It's the default recommendation for users who want Qwen3's reasoning upgrade without stepping up to 14B.

MMLU-Pro 56.73 is a large jump over Qwen2.5-7B's 36.5, making this one of the strongest dense 8B-class open models for reasoning tasks.

VRAM at each quantization

Numbers here are computed at 8k context. Because KV cache grows linearly with context length, expect higher totals at longer sequence lengths.

QuantWeightsKV cacheTotal
FP3232.0 GB1.21 GB37.2 GB
BF1616.0 GB1.21 GB19.3 GB
FP1616.0 GB1.21 GB19.3 GB
Q8_08.5 GB1.21 GB10.9 GB
Q6_K6.6 GB1.21 GB8.7 GB
Q5_K_Mrec5.7 GB1.21 GB7.7 GB
Q4_K_M4.9 GB1.21 GB6.8 GB
Q3_K_M3.9 GB1.21 GB5.7 GB
Q2_K3.0 GB1.21 GB4.8 GB
NVFP4cuda4.0 GB1.21 GB5.8 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Benchmarks

MMLU-Pro
56.73

GPUs that run Qwen3 8B natively (102)

Plus 2 GPUs that run it with CPU offload (slower)

Notes

Supports thinking and non-thinking modes.

Hugging Face ↗Ollama ↗Released 2025-04-29

Compare Qwen3 8B with other models

Continue reading

vram-guides8 min
Best LLMs for 6 GB VRAM (2026)
vram-guides10 min
Qwen Family Guide: 2.5 to 3.6

Frequently asked questions

What are the VRAM requirements for Qwen3 8B?
Qwen3 8B requires approximately 6.8 GB of VRAM at Q4_K_M quantization, 10.9 GB at Q8, and 19.3 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Qwen3 8B have?
Qwen3 8B has 8 billion parameters.
How capable is Qwen3 8B?
With an MMLU-Pro score of 56.73, Qwen3 8B delivers solid general-purpose performance suitable for most everyday tasks and professional use.
Can Qwen3 8B run on a 16 GB GPU?
Yes. Qwen3 8B needs 6.8 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 4070 Ti Super.
What is the smallest quantization for Qwen3 8B that fits in 24 GB of VRAM?
At BF16, Qwen3 8B needs 19.3 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Qwen3 8B locally?
A 16 GB GPU is enough. At Q4_K_M, Qwen3 8B needs 6.8 GB VRAM. Good options: RTX 4080 (16 GB), RTX 4070 Ti Super (16 GB).