Gemma 2 27B Instruct
Gemma 2 27B Instruct needs roughly 22.0 GB VRAM at Q4_K_M quantization (64.4 GB at FP16). 86 GPUs we track can run it fully in VRAM at 8k context.
86 GPUs run this natively · 21 with CPU offload
Google27.2B params8k contextGemmaCommercial use ok
Gemma 2 27B Instruct is a 27.2B parameter dense model developed by Google. June 2024 release with 8K context, short context but efficient architecture.
To run Gemma 2 27B Instruct locally: Q4_K_M ~16-18GB and fits on 24GB GPU with room to spare. Good mid-range option.
MMLU-Pro 38.0%, strong for its size. The 8K context keeps KV cache tiny even at full context.
VRAM at each quantization
Figures below assume 8k context; KV cache grows linearly as context length increases.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 108.8 GB | 3.09 GB | 125.3 GB |
| BF16 | 54.4 GB | 3.09 GB | 64.4 GB |
| FP16 | 54.4 GB | 3.09 GB | 64.4 GB |
| Q8_0 | 28.9 GB | 3.09 GB | 35.8 GB |
| Q6_K | 22.3 GB | 3.09 GB | 28.5 GB |
| Q5_K_M | 19.4 GB | 3.09 GB | 25.1 GB |
| Q4_K_Mrec | 16.6 GB | 3.09 GB | 22.0 GB |
| Q3_K_M | 13.1 GB | 3.09 GB | 18.1 GB |
| Q2_K | 10.4 GB | 3.09 GB | 15.1 GB |
| NVFP4cuda | 13.6 GB | 3.09 GB | 18.7 GB |
KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Gemma 2 27B Instruct natively (86)
- NVIDIA RTX 5090NVFP4 · 75 t/s
- NVIDIA RTX 5080Q2_K · 53.5 t/s
- NVIDIA RTX 5070 TiQ2_K · 50.5 t/s
- NVIDIA RTX 5060 Ti 16GBQ2_K · 27.1 t/s
- NVIDIA RTX 4090Q4_K_M · 40.1 t/s
Show 81 more
- NVIDIA RTX 4080Q2_K · 41.5 t/s
- NVIDIA RTX 4070 Ti SUPERQ2_K · 39.2 t/s
- NVIDIA RTX 4060 Ti 16GBQ2_K · 17.9 t/s
- NVIDIA RTX 3090Q4_K_M · 37.6 t/s
- NVIDIA RTX 3090 TiQ4_K_M · 40.1 t/s
- NVIDIA B300 288GBBF16 · 30.3 t/s
- NVIDIA B200 180GBBF16 · 30.3 t/s
- NVIDIA H200 141GBBF16 · 26 t/s
- NVIDIA H100 80GBBF16 · 22.5 t/s
- NVIDIA A100 80GBBF16 · 17.6 t/s
- NVIDIA A100 40GBQ8_0 · 26 t/s
- NVIDIA L40SQ8_0 · 17.5 t/s
- NVIDIA RTX A6000Q8_0 · 20 t/s
- NVIDIA RTX 4000 AdaQ3_K_M · 16.6 t/s
- NVIDIA RTX 4500 AdaQ4_K_M · 18.4 t/s
- NVIDIA RTX 5000 AdaQ6_K · 18.9 t/s
- NVIDIA RTX 6000 AdaQ8_0 · 24.6 t/s
- NVIDIA RTX Pro 6000BF16 · 25.5 t/s
- NVIDIA DGX Spark (128GB)BF16 · 4.1 t/s
- AMD Radeon RX 9070 XTQ2_K · 37.5 t/s
- AMD Radeon RX 7900 XTXQ4_K_M · 38.4 t/s
- AMD Radeon RX 7900 XTQ3_K_M · 38.9 t/s
- AMD Radeon RX 7900 GREQ2_K · 34.1 t/s
- AMD Radeon RX 7800 XTQ2_K · 36.7 t/s
- AMD Radeon RX 6800 XTQ2_K · 30.7 t/s
- AMD Radeon PRO W7800Q6_K · 18.9 t/s
- AMD Radeon PRO W7900Q8_0 · 22.3 t/s
- AMD Instinct MI300XBF16 · 26.9 t/s
- AMD Radeon AI PRO R9700 32GBQ6_K · 20.9 t/s
- AMD Strix Halo (128GB)BF16 · 3.9 t/s
- AMD Strix Halo (96GB)BF16 · 3.9 t/s
- AMD Strix Halo (64GB)Q8_0 · 6.9 t/s
- AMD Strix Halo (32GB)Q4_K_M · 11.1 t/s
- Apple M5 Ultra (512GB)BF16 · 16.7 t/s
- Apple M5 Ultra (256GB)BF16 · 16.7 t/s
- Apple M5 Ultra (96GB)BF16 · 16.7 t/s
- Apple M5 Max (128GB)BF16 · 8.5 t/s
- Apple M5 Max (64GB)Q8_0 · 15.3 t/s
- Apple M5 Max (48GB)Q8_0 · 15.3 t/s
- Apple M5 Max (36GB)Q5_K_M · 16.4 t/s
- Apple M5 Pro (64GB)Q8_0 · 7.7 t/s
- Apple M5 Pro (48GB)Q8_0 · 7.7 t/s
- Apple M5 Pro (24GB)Q2_K · 18.3 t/s
- Apple M5 (32GB)Q4_K_M · 6.2 t/s
- Apple M6 (32GB)Q4_K_M · 6.9 t/s
- Apple M4 Max (128GB)BF16 · 7.6 t/s
- Apple M4 Max (64GB)Q8_0 · 13.6 t/s
- Apple M4 Max (48GB)Q8_0 · 13.6 t/s
- Apple M4 Max (36GB)Q5_K_M · 14.6 t/s
- Apple M4 Pro (48GB)Q8_0 · 6.8 t/s
- Apple M4 Pro (24GB)Q2_K · 16.2 t/s
- Apple M4 (32GB)Q4_K_M · 4.9 t/s
- Apple M3 Ultra (512GB)BF16 · 11.4 t/s
- Apple M3 Ultra (256GB)BF16 · 11.4 t/s
- Apple M3 Ultra (96GB)BF16 · 11.4 t/s
- Apple M3 Max (128GB)BF16 · 5.6 t/s
- Apple M3 Max (96GB)BF16 · 4.2 t/s
- Apple M3 Max (64GB)Q8_0 · 10 t/s
- Apple M3 Max (48GB)Q8_0 · 10 t/s
- Apple M3 Max (36GB)Q5_K_M · 10.7 t/s
- Apple M3 Pro (36GB)Q5_K_M · 5.3 t/s
- Apple M3 (24GB)Q2_K · 5.9 t/s
- Apple M2 Ultra (192GB)BF16 · 11.1 t/s
- Apple M2 Ultra (64GB)Q8_0 · 20 t/s
- Apple M2 Max (96GB)BF16 · 5.6 t/s
- Apple M2 Max (64GB)Q8_0 · 10 t/s
- Apple M2 Max (32GB)Q4_K_M · 16.3 t/s
- Apple M2 Pro (32GB)Q4_K_M · 8.1 t/s
- Apple M2 (24GB)Q2_K · 5.9 t/s
- Apple M1 Ultra (128GB)BF16 · 11.1 t/s
- Apple M1 Ultra (64GB)Q8_0 · 20 t/s
- Apple M1 Max (64GB)Q8_0 · 10 t/s
- Apple M1 Max (32GB)Q4_K_M · 16.3 t/s
- Apple M1 Pro (32GB)Q4_K_M · 8.1 t/s
- Intel Arc Pro B70 32GBQ6_K · 19.9 t/s
- Intel Arc Pro B60 24GBQ4_K_M · 16.3 t/s
- Intel Arc Pro B50 16GBQ2_K · 14.1 t/s
- Intel Arc A770 16GBQ2_K · 33.3 t/s
- Intel Data Center GPU Max 1550BF16 · 22.3 t/s
- Intel Data Center GPU Max 1100Q8_0 · 22.4 t/s
- Intel Arc 140V (32GB)Q4_K_M · 6 t/s
Plus 21 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 5070NVFP4 · 4.3 t/s
- NVIDIA RTX 5060 Ti 8GBNVFP4 · 2.6 t/s
- NVIDIA RTX 5060NVFP4 · 2.6 t/s
- NVIDIA RTX 5050NVFP4 · 2.5 t/s
- NVIDIA RTX 4070 TiQ8_0 · 1.2 t/s
- NVIDIA RTX 4070 SUPERQ8_0 · 1.2 t/s
- NVIDIA RTX 4070Q8_0 · 1.2 t/s
- NVIDIA RTX 4060Q6_K · 1.3 t/s
- NVIDIA RTX 3080 10GBQ6_K · 1.5 t/s
- NVIDIA RTX 3060 12GBQ8_0 · 1.2 t/s
- Intel Arc B580 12GBQ8_0 · 1.2 t/s
- Intel Arc B570 10GBQ6_K · 1.5 t/s
- Intel Arc A770 8GBQ6_K · 1.4 t/s
- Intel Arc A750 8GBQ6_K · 1.4 t/s
- Intel Arc A580 8GBQ6_K · 1.4 t/s
- Intel Arc A380 6GBQ6_K · 1.2 t/s
- Intel Arc A310 4GBQ6_K · 1.1 t/s
- Intel Arc Pro A60 12GBQ8_0 · 1.2 t/s
- Intel Arc Pro A50 6GBQ6_K · 1.2 t/s
- Intel Arc Pro A40 6GBQ6_K · 1.2 t/s
- CPU only (system RAM)Q5_K_M · 1.8 t/s
No GPU? Rent an RTX 3090 from $0.22/hr ↗
Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.
Notes
Short 8k context: KV cache is tiny even at full context.
Compare Gemma 2 27B Instruct with other models
Frequently asked questions
- What are the VRAM requirements for Gemma 2 27B Instruct?
- Gemma 2 27B Instruct requires approximately 22.0 GB of VRAM at Q4_K_M quantization, 35.8 GB at Q8, and 64.4 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Gemma 2 27B Instruct have?
- Gemma 2 27B Instruct has 27.2 billion parameters.
- How capable is Gemma 2 27B Instruct?
- Gemma 2 27B Instruct has an MMLU-Pro score of 38, making it well-suited for lightweight tasks, prototyping, and resource-constrained environments.
- Can Gemma 2 27B Instruct run on a 16 GB GPU?
- No. At Q4_K_M, Gemma 2 27B Instruct needs 22.0 GB of VRAM, more than 16 GB. You will need a 24 GB GPU like the RTX 4090 or RTX 3090.
- Can Gemma 2 27B Instruct run on a 24 GB GPU?
- Yes. Gemma 2 27B Instruct fits in a 24 GB GPU at Q4_K_M, requiring 22.0 GB VRAM. GPUs with 24 GB include the RTX 4090, RTX 3090, and RTX 3090 Ti.
- What is the smallest quantization for Gemma 2 27B Instruct that fits in 24 GB of VRAM?
- At NVFP4, Gemma 2 27B Instruct needs 18.7 GB, the highest-quality quantization that fits in 24 GB of VRAM.
- What GPU do I need to run Gemma 2 27B Instruct locally?
- A 24 GB GPU is the minimum. At Q4_K_M, Gemma 2 27B Instruct needs 22.0 GB VRAM. Good options: RTX 4090 (24 GB), RTX 3090 (24 GB).
- Can I run Gemma 2 27B Instruct in the cloud?
- Yes. The cheapest RunPod setup we found that holds Gemma 2 27B Instruct entirely in GPU memory at 8k context is 1× RTX 3090 at Q4_K_M, at roughly 37.6 tokens per second and from $0.22 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.