CanItRun Logocanitrun.

Gemma 2 2B Instruct

Gemma 2 2B Instruct needs roughly 2.8 GB VRAM at Q4_K_M quantization (6.8 GB at FP16). 103 GPUs we track can run it fully in VRAM at 8k context.

103 GPUs run this natively · 1 with CPU offload

Google2.6B params8k contextGemmaCommercial use ok

Gemma 2 2B Instruct is a 2.6B parameter dense model developed by Google. Ultra-compact 2.6B model for edge deployment.

To run Gemma 2 2B Instruct locally: Q8_K_M ~2.5GB — runs on phones and integrated graphics.

Surprisingly capable for its size — MMLU-Pro 17.8% is strong at 2B scale.

VRAM at each quantization

Calculated at 8k context. Since KV cache scales linearly with context, longer sessions need more VRAM than shown here.

QuantWeightsKV cacheTotal
FP3210.4 GB0.87 GB12.6 GB
BF165.2 GB0.87 GB6.8 GB
FP165.2 GB0.87 GB6.8 GB
Q8_0rec2.8 GB0.87 GB4.1 GB
Q6_K2.1 GB0.87 GB3.4 GB
Q5_K_M1.9 GB0.87 GB3.0 GB
Q4_K_M1.6 GB0.87 GB2.8 GB
Q3_K_M1.3 GB0.87 GB2.4 GB
Q2_K1.0 GB0.87 GB2.1 GB
NVFP4cuda1.3 GB0.87 GB2.4 GB

Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.

Benchmarks

MMLU-Pro
17.8
IFEval
55.8
MATH
25

GPUs that run Gemma 2 2B Instruct natively (103)

Plus 1 GPUs that run it with CPU offload (slower)
Hugging Face ↗Ollama ↗Released 2024-07-31

Compare Gemma 2 2B Instruct with other models

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Gemma 2 2B Instruct?
Gemma 2 2B Instruct requires approximately 2.8 GB of VRAM at Q4_K_M quantization, 4.1 GB at Q8, and 6.8 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Gemma 2 2B Instruct have?
Gemma 2 2B Instruct has 2.6 billion parameters.
How capable is Gemma 2 2B Instruct?
Gemma 2 2B Instruct has an MMLU-Pro score of 17.8, making it well-suited for lightweight tasks, prototyping, and resource-constrained environments.
Can Gemma 2 2B Instruct run on a 16 GB GPU?
Yes. Gemma 2 2B Instruct needs 2.8 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 4070 Ti Super.
What is the smallest quantization for Gemma 2 2B Instruct that fits in 24 GB of VRAM?
At FP32, Gemma 2 2B Instruct needs 12.6 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Gemma 2 2B Instruct locally?
A 16 GB GPU is enough. At Q4_K_M, Gemma 2 2B Instruct needs 2.8 GB VRAM. Good options: RTX 4080 (16 GB), RTX 4070 Ti Super (16 GB).