CanItRun Logocanitrun.

Gemma 4 12B (Unified)

Gemma 4 12B (Unified) needs roughly 8.7 GB VRAM at Q4_K_M quantization (27.4 GB at FP16). 116 GPUs we track can run it fully in VRAM at 8k context.

116 GPUs run this natively · 2 with CPU offload

Google12B params256k contextApache 2.0Commercial use ok

Gemma 4 12B (Unified) is a 12B parameter dense model developed by Google. June 2026 release using an encoder-free 'Unified' architecture: image, audio, and video are projected directly into the transformer instead of routing through separate encoders. 256K native context, Apache 2.0.

To run Gemma 4 12B (Unified) locally: Q4_K_M needs ~7.3GB for weights (~8.7GB total at 8K context, since 40 of its 48 layers cap their KV cache at a 1,024-token window), comfortable even on a 12GB GPU, unlike the 26B and 31B siblings, which need a 24GB card at Q4_K_M.

MMLU-Pro 77.2 at 12B lands within reach of the 26B and 31B siblings, the best quality-per-GB in the Gemma 4 lineup.

VRAM at each quantization

Gemma 4 12B (Unified) natively supports a longer context window, but the table below is capped at 8k for comparability; its hybrid attention stack keeps KV cache growth well below linear.

QuantWeightsKV cacheTotal
FP3248.0 GB0.47 GB54.3 GB
BF1624.0 GB0.47 GB27.4 GB
FP1624.0 GB0.47 GB27.4 GB
Q8_012.8 GB0.47 GB14.8 GB
Q6_K9.8 GB0.47 GB11.6 GB
Q5_K_M8.5 GB0.47 GB10.1 GB
Q4_K_Mrec7.3 GB0.47 GB8.7 GB
Q3_K_M5.8 GB0.47 GB7.0 GB
Q2_K4.6 GB0.47 GB5.7 GB
NVFP4cuda6.0 GB0.47 GB7.3 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Benchmarks

MMLU-Pro
77.2

GPUs that run Gemma 4 12B (Unified) natively (116)

Show 111 more
Plus 2 GPUs that run it with CPU offload (slower)

No GPU? Rent an RTX 3080 10GB from $0.17/hr ↗

Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

Encoder-free 'unified' architecture projects image/audio/video directly into the transformer instead of using separate encoders. Close behind the 26B and 31B siblings on most benchmarks at under half the size.

Hugging Face ↗Ollama ↗Released 2026-06-03

Continue reading

vram-guides13 min
Best LLMs for 16 GB VRAM GPUs (2026)
vram-guides10 min
Best LLMs for 12 GB VRAM (2026): RTX 3060, 4070 & 5070
tutorials12 min
Getting Started with Local LLMs (2026): The Complete Beginner's Guide

Frequently asked questions

What are the VRAM requirements for Gemma 4 12B (Unified)?
Gemma 4 12B (Unified) requires approximately 8.7 GB of VRAM at Q4_K_M quantization, 14.8 GB at Q8, and 27.4 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Gemma 4 12B (Unified) have?
Gemma 4 12B (Unified) has 12 billion parameters.
How capable is Gemma 4 12B (Unified)?
Gemma 4 12B (Unified) achieves an MMLU-Pro score of 77.2, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
Can Gemma 4 12B (Unified) run on a 16 GB GPU?
Yes. Gemma 4 12B (Unified) needs 8.7 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 5070 Ti.
What is the smallest quantization for Gemma 4 12B (Unified) that fits in 24 GB of VRAM?
At NVFP4, Gemma 4 12B (Unified) needs 7.3 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Gemma 4 12B (Unified) locally?
A 16 GB GPU is enough. At Q4_K_M, Gemma 4 12B (Unified) needs 8.7 GB VRAM. Good options: RTX 4080 (16 GB), RTX 5070 Ti (16 GB).
Can I run Gemma 4 12B (Unified) in the cloud?
Yes. The cheapest RunPod setup we found that holds Gemma 4 12B (Unified) entirely in GPU memory at 8k context is 1× RTX 3080 10GB at Q4_K_M, at roughly 68.9 tokens per second and from $0.17 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.