CanItRun Logocanitrun.

Gemma 3 12B Instruct

Gemma 3 12B Instruct needs roughly 8.9 GB VRAM at Q4_K_M quantization (28.5 GB at FP16). 105 GPUs we track can run it fully in VRAM at 8k context.

105 GPUs run this natively · 2 with CPU offload

Google12.2B params128k contextGemmaCommercial use ok

Gemma 3 12B Instruct is a 12.2B parameter dense model developed by Google. Mid-size Gemma 3 with multimodal capabilities and 128K context.

To run Gemma 3 12B Instruct locally: Q5_K_M ~8-9GB — fits on 12GB GPUs comfortably.

12B sweet spot with vision support — balances quality and accessibility.

VRAM at each quantization

Assumes 8k context. KV cache grows linearly with context length.

QuantWeightsKV cacheTotal
FP3248.8 GB1.06 GB55.8 GB
BF1624.4 GB1.06 GB28.5 GB
FP1624.4 GB1.06 GB28.5 GB
Q8_012.2 GB1.06 GB14.8 GB
Q6_K10.0 GB1.06 GB12.4 GB
Q5_K_Mrec7.9 GB1.06 GB10.0 GB
Q4_K_M6.9 GB1.06 GB8.9 GB
Q3_K_M5.3 GB1.06 GB7.1 GB
Q2_K4.0 GB1.06 GB5.7 GB
NVFP4cuda6.1 GB1.06 GB8.0 GB

KV cache shown at 8k context (FP16). NVFP4 requires a CUDA GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.

Benchmarks

MMLU-Pro
60.6

GPUs that run Gemma 3 12B Instruct natively (105)

Plus 2 GPUs that run it with CPU offload (slower)
Hugging Face ↗Ollama ↗Released 2025-03-12

Compare Gemma 3 12B Instruct with other models

Frequently asked questions

What are the VRAM requirements for Gemma 3 12B Instruct?
Gemma 3 12B Instruct requires approximately 8.9 GB of VRAM at Q4_K_M quantization, 14.8 GB at Q8, and 28.5 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Gemma 3 12B Instruct have?
Gemma 3 12B Instruct has 12.2 billion parameters.
How capable is Gemma 3 12B Instruct?
With an MMLU-Pro score of 60.6, Gemma 3 12B Instruct delivers solid general-purpose performance suitable for most everyday tasks and professional use.
Can Gemma 3 12B Instruct run on a 16 GB GPU?
Yes. Gemma 3 12B Instruct needs 8.9 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 4070 Ti Super.
What is the smallest quantization for Gemma 3 12B Instruct that fits in 24 GB of VRAM?
At NVFP4, Gemma 3 12B Instruct needs 8.0 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Gemma 3 12B Instruct locally?
A 16 GB GPU is enough. At Q4_K_M, Gemma 3 12B Instruct needs 8.9 GB VRAM. Good options: RTX 4080 (16 GB), RTX 4070 Ti Super (16 GB).