Gemma 2 27B Instruct
Gemma 2 27B Instruct needs roughly 22.0 GB VRAM at Q4_K_M quantization (64.4 GB at FP16). 75 GPUs we track can run it fully in VRAM at 8k context.
75 GPUs run this natively · 19 with CPU offload
Google27.2B params8k contextGemmaCommercial use ok
Gemma 2 27B Instruct is a 27.2B parameter dense model developed by Google. June 2024 release with 8K context — short context but efficient architecture.
To run Gemma 2 27B Instruct locally: Q4_K_M ~16-18GB — fits on 24GB GPU with room to spare. Good mid-range option.
MMLU-Pro 38.0%, strong for its size. The 8K context keeps KV cache tiny even at full context.
VRAM at each quantization
Figures below assume 8k context; KV cache grows linearly as context length increases.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 108.8 GB | 3.09 GB | 125.3 GB |
| BF16 | 54.4 GB | 3.09 GB | 64.4 GB |
| FP16 | 54.4 GB | 3.09 GB | 64.4 GB |
| Q8_0 | 28.9 GB | 3.09 GB | 35.8 GB |
| Q6_K | 22.3 GB | 3.09 GB | 28.5 GB |
| Q5_K_M | 19.4 GB | 3.09 GB | 25.1 GB |
| Q4_K_Mrec | 16.6 GB | 3.09 GB | 22.0 GB |
| Q3_K_M | 13.1 GB | 3.09 GB | 18.1 GB |
| Q2_K | 10.4 GB | 3.09 GB | 15.1 GB |
| NVFP4cuda | 13.6 GB | 3.09 GB | 18.7 GB |
KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Gemma 2 27B Instruct natively (75)
- NVIDIA RTX 5090NVFP4 · 69.8 t/s
- NVIDIA RTX 5080Q2_K · 46.4 t/s
- NVIDIA RTX 5070 TiQ2_K · 43.3 t/s
- NVIDIA RTX 5060 Ti 16GBQ2_K · 21.7 t/s
- NVIDIA RTX 4090NVFP4 · 39.3 t/s
- NVIDIA RTX 4080Q2_K · 34.7 t/s
- NVIDIA RTX 4060 Ti 16GBQ2_K · 13.9 t/s
- NVIDIA RTX 3090NVFP4 · 36.5 t/s
- NVIDIA RTX 3090 TiNVFP4 · 39.3 t/s
- NVIDIA H100 80GBBF16 · 37.9 t/s
- NVIDIA A100 80GBBF16 · 23.1 t/s
- NVIDIA A100 40GBNVFP4 · 60.6 t/s
- NVIDIA L40SNVFP4 · 33.7 t/s
- NVIDIA RTX A6000NVFP4 · 29.9 t/s
- NVIDIA RTX 4000 AdaNVFP4 · 12.5 t/s
- NVIDIA RTX 4500 AdaNVFP4 · 16.8 t/s
- NVIDIA RTX 5000 AdaNVFP4 · 22.4 t/s
- NVIDIA RTX 6000 AdaNVFP4 · 37.4 t/s
- NVIDIA RTX Pro 6000BF16 · 15.2 t/s
- NVIDIA DGX Spark (128GB)BF16 · 3.1 t/s
- AMD Radeon RX 7900 XTXQ4_K_M · 31.8 t/s
- AMD Radeon RX 7900 XTQ3_K_M · 32.2 t/s
- AMD Radeon RX 7900 GREQ2_K · 27.8 t/s
- AMD Radeon RX 6800 XTQ2_K · 24.7 t/s
- AMD Radeon PRO W7800Q6_K · 14.7 t/s
- AMD Radeon PRO W7900Q8_0 · 17.5 t/s
- AMD Instinct MI300XFP32 · 30.8 t/s
- AMD Radeon AI Pro 9700 32GBQ6_K · 16.4 t/s
- AMD Strix Halo (128GB)BF16 · 2.9 t/s
- AMD Strix Halo (96GB)BF16 · 2.9 t/s
- AMD Strix Halo (64GB)Q8_0 · 5.2 t/s
- Apple M5 Max (128GB)BF16 · 8.5 t/s
- Apple M5 Max (64GB)Q8_0 · 15.3 t/s
- Apple M5 Max (48GB)Q8_0 · 15.3 t/s
- Apple M5 Pro (48GB)Q8_0 · 7.7 t/s
- Apple M5 Pro (36GB)Q5_K_M · 10.9 t/s
- Apple M5 Pro (24GB)Q2_K · 18.3 t/s
- Apple M5 (32GB)Q4_K_M · 6.2 t/s
- Apple M4 Ultra (384GB)FP32 · 7.8 t/s
- Apple M4 Ultra (192GB)FP32 · 7.8 t/s
- Apple M4 Max (128GB)BF16 · 7.6 t/s
- Apple M4 Max (96GB)BF16 · 7.6 t/s
- Apple M4 Max (64GB)Q8_0 · 13.6 t/s
- Apple M4 Max (48GB)Q8_0 · 13.6 t/s
- Apple M4 Pro (48GB)Q8_0 · 6.8 t/s
- Apple M4 Pro (24GB)Q2_K · 16.2 t/s
- Apple M4 (32GB)Q4_K_M · 4.9 t/s
- Apple M3 Ultra (512GB)FP32 · 5.9 t/s
- Apple M3 Ultra (256GB)FP32 · 5.9 t/s
- Apple M3 Ultra (96GB)BF16 · 11.4 t/s
- Apple M3 Max (128GB)BF16 · 5.6 t/s
- Apple M3 Max (96GB)BF16 · 5.6 t/s
- Apple M3 Max (64GB)Q8_0 · 10 t/s
- Apple M3 Max (48GB)Q8_0 · 10 t/s
- Apple M3 Max (36GB)Q5_K_M · 14.3 t/s
- Apple M3 Pro (36GB)Q5_K_M · 5.3 t/s
- Apple M3 (24GB)Q2_K · 5.9 t/s
- Apple M2 Ultra (384GB)FP32 · 5.7 t/s
- Apple M2 Ultra (192GB)FP32 · 5.7 t/s
- Apple M2 Max (96GB)BF16 · 5.6 t/s
- Apple M2 Max (64GB)Q8_0 · 10 t/s
- Apple M2 Max (32GB)Q4_K_M · 16.3 t/s
- Apple M2 Pro (32GB)Q4_K_M · 8.1 t/s
- Apple M2 (24GB)Q2_K · 5.9 t/s
- Apple M1 Ultra (128GB)BF16 · 11.1 t/s
- Apple M1 Ultra (64GB)Q8_0 · 20 t/s
- Apple M1 Max (64GB)Q8_0 · 10 t/s
- Apple M1 Max (32GB)Q4_K_M · 16.3 t/s
- Apple M1 Pro (32GB)Q4_K_M · 8.1 t/s
- Intel Arc Pro B70 24GBQ4_K_M · 15.1 t/s
- Intel Arc Pro B60 24GBQ4_K_M · 12.6 t/s
- Intel Arc A770 16GBQ2_K · 27.1 t/s
- Intel Data Center GPU Max 1550BF16 · 37 t/s
- Intel Data Center GPU Max 1100Q8_0 · 25 t/s
- Intel Arc 140V (32GB)Q4_K_M · 4.5 t/s
Plus 19 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 5070NVFP4 · 4.2 t/s
- NVIDIA RTX 5060NVFP4 · 2.6 t/s
- NVIDIA RTX 5050NVFP4 · 2.5 t/s
- NVIDIA RTX 4070 TiNVFP4 · 4.1 t/s
- NVIDIA RTX 4070NVFP4 · 4.1 t/s
- NVIDIA RTX 4060NVFP4 · 2.5 t/s
- NVIDIA RTX 3080 10GBNVFP4 · 3.3 t/s
- NVIDIA RTX 3060 12GBNVFP4 · 3.8 t/s
- Intel Arc B580 12GBQ8_0 · 1.2 t/s
- Intel Arc B570 10GBQ6_K · 1.5 t/s
- Intel Arc A770 8GBQ6_K · 1.4 t/s
- Intel Arc A750 8GBQ6_K · 1.4 t/s
- Intel Arc A580 8GBQ6_K · 1.4 t/s
- Intel Arc A380 6GBQ6_K · 1.2 t/s
- Intel Arc A310 4GBQ6_K · 1.1 t/s
- Intel Arc Pro A60 12GBQ8_0 · 1.1 t/s
- Intel Arc Pro A50 6GBQ6_K · 1.2 t/s
- Intel Arc Pro A40 6GBQ6_K · 1.2 t/s
- CPU only (system RAM)Q5_K_M · 1.8 t/s
Notes
Short 8k context — KV cache is tiny even at full context.
Compare Gemma 2 27B Instruct with other models
Frequently asked questions
- What are the VRAM requirements for Gemma 2 27B Instruct?
- Gemma 2 27B Instruct requires approximately 22.0 GB of VRAM at Q4_K_M quantization, 35.8 GB at Q8, and 64.4 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Gemma 2 27B Instruct have?
- Gemma 2 27B Instruct has 27.2 billion parameters.
- How capable is Gemma 2 27B Instruct?
- Gemma 2 27B Instruct has an MMLU-Pro score of 38, making it well-suited for lightweight tasks, prototyping, and resource-constrained environments.
- Can Gemma 2 27B Instruct run on a 16 GB GPU?
- No. At Q4_K_M, Gemma 2 27B Instruct needs 22.0 GB of VRAM — more than 16 GB. You will need a 24 GB GPU like the RTX 4090 or RTX 3090.
- Can Gemma 2 27B Instruct run on a 24 GB GPU?
- Yes. Gemma 2 27B Instruct fits in a 24 GB GPU at Q4_K_M, requiring 22.0 GB VRAM. GPUs with 24 GB include the RTX 4090, RTX 3090, and RTX 3090 Ti.
- What is the smallest quantization for Gemma 2 27B Instruct that fits in 24 GB of VRAM?
- At NVFP4, Gemma 2 27B Instruct needs 18.7 GB — the highest-quality quantization that fits in 24 GB of VRAM.
- What GPU do I need to run Gemma 2 27B Instruct locally?
- A 24 GB GPU is the minimum. At Q4_K_M, Gemma 2 27B Instruct needs 22.0 GB VRAM. Good options: RTX 4090 (24 GB), RTX 3090 (24 GB).