Gemma 2 2B Instruct
Gemma 2 2B Instruct needs roughly 2.8 GB VRAM at Q4_K_M quantization (6.8 GB at FP16). 117 GPUs we track can run it fully in VRAM at 8k context.
117 GPUs run this natively · 1 with CPU offload
Google2.6B params8k contextGemmaCommercial use ok
Gemma 2 2B Instruct is a 2.6B parameter dense model developed by Google. Ultra-compact 2.6B model for edge deployment.
To run Gemma 2 2B Instruct locally: Q8_K_M ~2.5GB and runs on phones and integrated graphics.
Surprisingly capable for its size: MMLU-Pro 17.8% is strong at 2B scale.
VRAM at each quantization
Calculated at 8k context. Since KV cache scales linearly with context, longer sessions need more VRAM than shown here.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 10.4 GB | 0.87 GB | 12.6 GB |
| BF16 | 5.2 GB | 0.87 GB | 6.8 GB |
| FP16 | 5.2 GB | 0.87 GB | 6.8 GB |
| Q8_0rec | 2.8 GB | 0.87 GB | 4.1 GB |
| Q6_K | 2.1 GB | 0.87 GB | 3.4 GB |
| Q5_K_M | 1.9 GB | 0.87 GB | 3.0 GB |
| Q4_K_M | 1.6 GB | 0.87 GB | 2.8 GB |
| Q3_K_M | 1.3 GB | 0.87 GB | 2.4 GB |
| Q2_K | 1.0 GB | 0.87 GB | 2.1 GB |
| NVFP4cuda | 1.3 GB | 0.87 GB | 2.4 GB |
Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.
Benchmarks
GPUs that run Gemma 2 2B Instruct natively (117)
- NVIDIA RTX 5090BF16 · 184.8 t/s
- NVIDIA RTX 5080BF16 · 114.3 t/s
- NVIDIA RTX 5070 TiBF16 · 108 t/s
- NVIDIA RTX 5070BF16 · 84.5 t/s
- NVIDIA RTX 5060 Ti 16GBBF16 · 59 t/s
Show 112 more
- NVIDIA RTX 5060 Ti 8GBBF16 · 59 t/s
- NVIDIA RTX 5060BF16 · 59 t/s
- NVIDIA RTX 5050BF16 · 43.2 t/s
- NVIDIA RTX 4090BF16 · 119 t/s
- NVIDIA RTX 4080BF16 · 89.4 t/s
- NVIDIA RTX 4070 Ti SUPERBF16 · 84.5 t/s
- NVIDIA RTX 4070 TiBF16 · 65.6 t/s
- NVIDIA RTX 4070 SUPERBF16 · 65.6 t/s
- NVIDIA RTX 4070BF16 · 65.6 t/s
- NVIDIA RTX 4060 Ti 16GBBF16 · 39.2 t/s
- NVIDIA RTX 4060BF16 · 37.1 t/s
- NVIDIA RTX 3090BF16 · 112 t/s
- NVIDIA RTX 3090 TiBF16 · 119 t/s
- NVIDIA RTX 3080 10GBBF16 · 94 t/s
- NVIDIA RTX 3060 12GBBF16 · 48.2 t/s
- NVIDIA B300 288GBBF16 · 210.2 t/s
- NVIDIA B200 180GBBF16 · 210.2 t/s
- NVIDIA H200 141GBBF16 · 187.5 t/s
- NVIDIA H100 80GBBF16 · 167.9 t/s
- NVIDIA A100 80GBBF16 · 137.4 t/s
- NVIDIA A100 40GBBF16 · 120 t/s
- NVIDIA L40SBF16 · 84.2 t/s
- NVIDIA RTX A6000BF16 · 94.8 t/s
- NVIDIA RTX 4000 AdaBF16 · 43.2 t/s
- NVIDIA RTX 4500 AdaBF16 · 57 t/s
- NVIDIA RTX 5000 AdaBF16 · 73.9 t/s
- NVIDIA RTX 6000 AdaBF16 · 114.3 t/s
- NVIDIA RTX Pro 6000BF16 · 184.8 t/s
- NVIDIA DGX Spark (128GB)BF16 · 37.3 t/s
- AMD Radeon RX 9070 XTBF16 · 81 t/s
- AMD Radeon RX 7900 XTXBF16 · 114.3 t/s
- AMD Radeon RX 7900 XTBF16 · 98.2 t/s
- AMD Radeon RX 7900 GREBF16 · 73.9 t/s
- AMD Radeon RX 7800 XTBF16 · 79.2 t/s
- AMD Radeon RX 6800 XTBF16 · 66.5 t/s
- AMD Radeon PRO W7800BF16 · 73.9 t/s
- AMD Radeon PRO W7900BF16 · 104.7 t/s
- AMD Instinct MI300XBF16 · 192.4 t/s
- AMD Radeon AI PRO R9700 32GBBF16 · 81 t/s
- AMD Strix Halo (128GB)BF16 · 35.1 t/s
- AMD Strix Halo (96GB)BF16 · 35.1 t/s
- AMD Strix Halo (64GB)BF16 · 35.1 t/s
- AMD Strix Halo (32GB)BF16 · 35.1 t/s
- Apple M5 Ultra (512GB)BF16 · 158.1 t/s
- Apple M5 Ultra (256GB)BF16 · 158.1 t/s
- Apple M5 Ultra (96GB)BF16 · 158.1 t/s
- Apple M5 Max (128GB)BF16 · 80.9 t/s
- Apple M5 Max (64GB)BF16 · 80.9 t/s
- Apple M5 Max (48GB)BF16 · 80.9 t/s
- Apple M5 Max (36GB)BF16 · 60.6 t/s
- Apple M5 Pro (64GB)BF16 · 40.4 t/s
- Apple M5 Pro (48GB)BF16 · 40.4 t/s
- Apple M5 Pro (24GB)BF16 · 40.4 t/s
- Apple M5 (32GB)BF16 · 20.2 t/s
- Apple M5 (16GB)BF16 · 20.2 t/s
- Apple M6 (32GB)BF16 · 22.4 t/s
- Apple M6 (16GB)BF16 · 22.4 t/s
- Apple M4 Max (128GB)BF16 · 71.9 t/s
- Apple M4 Max (64GB)BF16 · 71.9 t/s
- Apple M4 Max (48GB)BF16 · 71.9 t/s
- Apple M4 Max (36GB)BF16 · 54 t/s
- Apple M4 Pro (48GB)BF16 · 36 t/s
- Apple M4 Pro (24GB)BF16 · 36 t/s
- Apple M4 (32GB)BF16 · 15.8 t/s
- Apple M4 (16GB)BF16 · 15.8 t/s
- Apple M3 Ultra (512GB)BF16 · 107.9 t/s
- Apple M3 Ultra (256GB)BF16 · 107.9 t/s
- Apple M3 Ultra (96GB)BF16 · 107.9 t/s
- Apple M3 Max (128GB)BF16 · 52.7 t/s
- Apple M3 Max (96GB)BF16 · 39.5 t/s
- Apple M3 Max (64GB)BF16 · 52.7 t/s
- Apple M3 Max (48GB)BF16 · 52.7 t/s
- Apple M3 Max (36GB)BF16 · 39.5 t/s
- Apple M3 Pro (36GB)BF16 · 19.8 t/s
- Apple M3 Pro (18GB)BF16 · 19.8 t/s
- Apple M3 (24GB)BF16 · 13.2 t/s
- Apple M3 (16GB)BF16 · 13.2 t/s
- Apple M2 Ultra (192GB)BF16 · 105.4 t/s
- Apple M2 Ultra (64GB)BF16 · 105.4 t/s
- Apple M2 Max (96GB)BF16 · 52.7 t/s
- Apple M2 Max (64GB)BF16 · 52.7 t/s
- Apple M2 Max (32GB)BF16 · 52.7 t/s
- Apple M2 Pro (32GB)BF16 · 26.3 t/s
- Apple M2 Pro (16GB)BF16 · 26.3 t/s
- Apple M2 (24GB)BF16 · 13.2 t/s
- Apple M2 (16GB)BF16 · 13.2 t/s
- Apple M1 Ultra (128GB)BF16 · 105.4 t/s
- Apple M1 Ultra (64GB)BF16 · 105.4 t/s
- Apple M1 Max (64GB)BF16 · 52.7 t/s
- Apple M1 Max (32GB)BF16 · 52.7 t/s
- Apple M1 Pro (32GB)BF16 · 26.3 t/s
- Apple M1 Pro (16GB)BF16 · 26.3 t/s
- Apple M1 (16GB)BF16 · 9 t/s
- Intel Arc B580 12GBBF16 · 59.9 t/s
- Intel Arc B570 10GBBF16 · 50.7 t/s
- Intel Arc Pro B70 32GBBF16 · 77.5 t/s
- Intel Arc Pro B60 24GBBF16 · 50.7 t/s
- Intel Arc Pro B50 16GBBF16 · 30.9 t/s
- Intel Arc A770 16GBBF16 · 72 t/s
- Intel Arc A770 8GBBF16 · 66.5 t/s
- Intel Arc A750 8GBBF16 · 66.5 t/s
- Intel Arc A580 8GBBF16 · 66.5 t/s
- Intel Arc A380 6GBQ8_0 · 42.1 t/s
- Intel Arc A310 4GBQ6_K · 34.3 t/s
- Intel Arc Pro A60 12GBBF16 · 51.2 t/s
- Intel Arc Pro A50 6GBQ8_0 · 43.3 t/s
- Intel Arc Pro A40 6GBQ8_0 · 43.3 t/s
- Intel Data Center GPU Max 1550BF16 · 166.6 t/s
- Intel Data Center GPU Max 1100BF16 · 105.1 t/s
- Intel Arc 140V (32GB)BF16 · 19.3 t/s
- Intel Arc 140V (16GB)BF16 · 19.3 t/s
- Intel Arc 130V (16GB)BF16 · 19.3 t/s
Plus 1 GPU that runs it with CPU offload (slower)
- CPU only (system RAM)BF16 · 6.6 t/s
No GPU? Rent an RTX 3080 10GB from $0.17/hr ↗
Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.
Compare Gemma 2 2B Instruct with other models
Frequently asked questions
- What are the VRAM requirements for Gemma 2 2B Instruct?
- Gemma 2 2B Instruct requires approximately 2.8 GB of VRAM at Q4_K_M quantization, 4.1 GB at Q8, and 6.8 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Gemma 2 2B Instruct have?
- Gemma 2 2B Instruct has 2.6 billion parameters.
- How capable is Gemma 2 2B Instruct?
- Gemma 2 2B Instruct has an MMLU-Pro score of 17.8, making it well-suited for lightweight tasks, prototyping, and resource-constrained environments.
- Can Gemma 2 2B Instruct run on a 16 GB GPU?
- Yes. Gemma 2 2B Instruct needs 2.8 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 5070 Ti.
- What is the smallest quantization for Gemma 2 2B Instruct that fits in 24 GB of VRAM?
- At BF16, Gemma 2 2B Instruct needs 6.8 GB, the highest-quality quantization that fits in 24 GB of VRAM.
- What GPU do I need to run Gemma 2 2B Instruct locally?
- A 16 GB GPU is enough. At Q4_K_M, Gemma 2 2B Instruct needs 2.8 GB VRAM. Good options: RTX 4080 (16 GB), RTX 5070 Ti (16 GB).
- Can I run Gemma 2 2B Instruct in the cloud?
- Yes. The cheapest RunPod setup we found that holds Gemma 2 2B Instruct entirely in GPU memory at 8k context is 1× RTX 3080 10GB at BF16, at roughly 94 tokens per second and from $0.17 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.