CanItRun Logocanitrun.

GPT-OSS 20B

GPT-OSS 20B needs roughly 14.6 GB VRAM at Q4_K_M quantization (47.3 GB at FP16). 84 GPUs we track can run it fully in VRAM at 8k context.

84 GPUs run this natively · 21 with CPU offload

OpenAI21B params3.6B active (MoE)128k contextApache 2.0Commercial use ok

GPT-OSS 20B is a Mixture of Experts (MoE) model with 21B total parameters but only 3.6B active per token developed by OpenAI. Released 5 August 2025, 21B MoE with 3.6B active (32 experts, top-4 routed per token), the smaller sibling of GPT-OSS 120B, sharing the same alternating 128-token sliding-window / full-attention design. Matches o3-mini on key benchmarks.

To run GPT-OSS 20B locally: Same native-MXFP4-experts story as the 120B at a much smaller scale: real GGUF builds cluster 11.5-13.8GB regardless of nominal quant (bartowski/openai_gpt-oss-20b-GGUF), comfortably inside 16GB of VRAM, matching OpenAI's own claim that it runs in 16GB of memory. As a MoE model, inference speed depends on active parameters (3.6B) rather than total size.

GPQA Diamond 71.5% at 21B scale, exceptional reasoning efficiency, with the same configurable low/medium/high reasoning effort as the 120B.

VRAM at each quantization

Figures below assume 8k context. This model's hybrid attention stack caches only some layers, so KV cache grows much slower than linearly as context increases.

QuantWeightsKV cacheTotal
FP3284.0 GB0.20 GB94.3 GB
BF1642.0 GB0.20 GB47.3 GB
FP1642.0 GB0.20 GB47.3 GB
Q8_022.3 GB0.20 GB25.2 GB
Q6_K17.2 GB0.20 GB19.5 GB
Q5_K_M14.9 GB0.20 GB17.0 GB
Q4_K_Mrec12.8 GB0.20 GB14.6 GB
Q3_K_M11.5 GB0.20 GB13.1 GB
Q2_K11.5 GB0.20 GB13.1 GB
NVFP4cuda11.5 GB0.20 GB13.1 GB

Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.

Benchmarks

MMLU-Pro
67.9
GPQA Diamond
71.5

GPUs that run GPT-OSS 20B natively (84)

Show 79 more
Plus 21 GPUs that run it with CPU offload (slower)

Notes

Smaller sibling of GPT-OSS 120B, same MoE design (32 experts, top-4 routed) and alternating 128-token sliding-window attention. Matches o3-mini on key benchmarks; OpenAI's native MXFP4 build needs roughly 12-14GB, comfortably inside 16GB of VRAM.

Hugging Face ↗Released 2025-08-05

Continue reading

vram-guides9 min
Best LLMs for 16 GB VRAM GPUs (2026)
vram-guides8 min
Best LLMs for 12 GB VRAM (2026)

Frequently asked questions

What are the VRAM requirements for GPT-OSS 20B?
GPT-OSS 20B requires approximately 14.6 GB of VRAM at Q4_K_M quantization, 25.2 GB at Q8, and 47.3 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does GPT-OSS 20B have?
GPT-OSS 20B has 21 billion total parameters, but only 3.6 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
How capable is GPT-OSS 20B?
With an MMLU-Pro score of 67.86, GPT-OSS 20B delivers solid general-purpose performance suitable for most everyday tasks and professional use.
Can GPT-OSS 20B run on a 16 GB GPU?
Yes. GPT-OSS 20B needs 14.6 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 5070 Ti.
What is the smallest quantization for GPT-OSS 20B that fits in 24 GB of VRAM?
At NVFP4, GPT-OSS 20B needs 13.1 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run GPT-OSS 20B locally?
A 16 GB GPU is enough. At Q4_K_M, GPT-OSS 20B needs 14.6 GB VRAM. Good options: RTX 4080 (16 GB), RTX 5070 Ti (16 GB).