CanItRun Logocanitrun.

Gemma 3 12B Instruct

Gemma 3 12B Instruct needs roughly 9.5 GB VRAM at Q4_K_M quantization (28.5 GB at FP16). 99 GPUs we track can run it fully in VRAM at 8k context.

99 GPUs run this natively · 5 with CPU offload

Google12.2B params128k contextGemmaCommercial use ok

Gemma 3 12B Instruct is a 12.2B parameter dense model developed by Google. Mid-size Gemma 3 with multimodal capabilities and 128K context.

To run Gemma 3 12B Instruct locally: Q5_K_M ~8-9GB — fits on 12GB GPUs comfortably.

12B sweet spot with vision support — balances quality and accessibility.

VRAM at each quantization

Numbers here are computed at 8k context. Because KV cache grows linearly with context length, expect higher totals at longer sequence lengths.

QuantWeightsKV cacheTotal
FP3248.8 GB1.06 GB55.8 GB
BF1624.4 GB1.06 GB28.5 GB
FP1624.4 GB1.06 GB28.5 GB
Q8_013.0 GB1.06 GB15.7 GB
Q6_K10.0 GB1.06 GB12.4 GB
Q5_K_Mrec8.7 GB1.06 GB10.9 GB
Q4_K_M7.4 GB1.06 GB9.5 GB
Q3_K_M5.9 GB1.06 GB7.8 GB
Q2_K4.7 GB1.06 GB6.4 GB
NVFP4cuda6.1 GB1.06 GB8.0 GB

Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.

Benchmarks

MMLU-Pro
60.6

GPUs that run Gemma 3 12B Instruct natively (99)

Plus 5 GPUs that run it with CPU offload (slower)
Hugging Face ↗Ollama ↗Released 2025-03-12

Compare Gemma 3 12B Instruct with other models

How to run Gemma 3 12B Instruct locally

816244880160320

Q5_K_M needs 10.9 GB — fits a single high-end consumer GPU (24 GB).

Ollama

ollama run gemma3:12b

llama.cpp

./llama-cli -m gemma-3-12b-it.Q5_K_M.gguf -c 8192 -ngl 99

LM Studio: Search for 'Gemma 3 12B' in LM Studio. The Q5_K_M variant runs well on 12-16 GB GPUs. Supports both text and image inputs.

Why this quantization? At 12.2B parameters, Q5_K_M uses roughly 8 GB of VRAM for weights, fitting comfortably on a 12-16 GB GPU with room for the KV cache. The 5:1 local/global attention pattern keeps cache overhead manageable. Q5 preserves the model's solid MMLU-Pro score (60.6) better than Q4 would, and the additional VRAM cost over Q4 is only about 1 GB.

Who is Gemma 3 12B Instruct for?

Users with mid-range GPUs (12-16 GB) who want multimodal capabilities without stepping up to a 24 GB card. A good middle ground for developers who need both text and vision understanding on consumer hardware.

Best for

  • Image-to-text tasks like describing photos, reading charts, or parsing screenshots
  • General chat and writing assistance on mid-range hardware
  • Multilingual text generation and translation
  • Building multimodal applications with a manageable model footprint

Not ideal for

  • Heavy reasoning or math tasks -- Phi-4 14B significantly outperforms at a similar size
  • Code-specialized tasks where Qwen 2.5 Coder is purpose-built
  • Production deployments requiring frontier-level accuracy

Continue reading

vram-guides9 min
Best LLMs for 16 GB VRAM GPUs (2026)
vram-guides8 min
Best LLMs for 12 GB VRAM (2026)
hardware9 min
Best GPUs Under $500 for LLMs (2026)

Frequently asked questions

What are the VRAM requirements for Gemma 3 12B Instruct?
Gemma 3 12B Instruct requires approximately 9.5 GB of VRAM at Q4_K_M quantization, 15.7 GB at Q8, and 28.5 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Gemma 3 12B Instruct have?
Gemma 3 12B Instruct has 12.2 billion parameters.
How capable is Gemma 3 12B Instruct?
With an MMLU-Pro score of 60.6, Gemma 3 12B Instruct delivers solid general-purpose performance suitable for most everyday tasks and professional use.
Can Gemma 3 12B Instruct run on a 16 GB GPU?
Yes. Gemma 3 12B Instruct needs 9.5 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 4070 Ti Super.
What is the smallest quantization for Gemma 3 12B Instruct that fits in 24 GB of VRAM?
At NVFP4, Gemma 3 12B Instruct needs 8.0 GB — the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Gemma 3 12B Instruct locally?
A 16 GB GPU is enough. At Q4_K_M, Gemma 3 12B Instruct needs 9.5 GB VRAM. Good options: RTX 4080 (16 GB), RTX 4070 Ti Super (16 GB).