CanItRun Logocanitrun.

Gemma 4 26B (MoE)

Gemma 4 26B (MoE) needs roughly 17.6 GB VRAM at Q4_K_M quantization (56.9 GB at FP16). 91 GPUs we track can run it fully in VRAM at 8k context.

91 GPUs run this natively · 14 with CPU offload

Google25.2B params3.8B active (MoE)256k contextApache 2.0Commercial use ok
Q4_K_M total
17.6 GB
at 8k context
Smallest GPU
20 GB
NVIDIA RTX 4000 Ada, at Q4_K_M
KV cache, full context
5.6 GB
5 of 30 layers grow with context, 25 capped at 1,024 tokens
Inputs
Text · Image
Apache 2.0, released April 2026
Active experts
8 of 128 + 1 shared
~3.8B active per token
Attention pattern
5 local : 1 global
1,024-token sliding window

Gemma 4 26B (MoE) is a Mixture of Experts (MoE) model with 25.2B total parameters but only 3.8B active per token developed by Google. Released 2 April 2026 under Apache 2.0, this MoE variant activates roughly 3.8B of its 25.2B parameters per token by routing through 8 of 128 experts plus one always-on shared expert (Google brands the release "26B"; its own model card lists 25.2B total). Its 30 attention layers keep Gemma 3's local:global pattern: five sliding-window layers (1,024-token lookback) for every one full-attention layer. Context is 262,144 tokens natively, and the same checkpoint accepts text and image input.

To run Gemma 4 26B (MoE) locally: A Q4_K_M GGUF is roughly 15.3GB of weights, so about 17.6GB in total at 8k context; a 20GB+ card is the practical floor, since the weights alone already exceed a 16GB budget once overhead is added. The sliding-window attention is what keeps long context affordable: filling the full 262,144-token window costs only about 23.4GB total, comfortably inside a 32GB card (a 24GB card needs a sliver of CPU offload right at the top of that window). Vision uses a separate encoder, and Google's own GGUF quants (e.g. via Unsloth) are the most reliable route for llama.cpp/Ollama. As a MoE model, inference speed depends on active parameters (3.8B) rather than total size.

Google reports MMLU-Pro 82.6, GPQA Diamond 82.3, and LiveCodeBench v6 77.1, a profile that lands within a few points of the dense 31B sibling despite loading fewer active parameters per token. These are the vendor's own published figures; independent replication was still thin shortly after launch.

Five sliding-window layers for every one full-attention layer

Gemma 4 26B routes each token through 8 of 128 experts, plus one "shared" expert that runs every time: that routing is what keeps only about 3.8B parameters active per token even though all 26B have to sit in memory to make the routing fast. Separately, its 30 attention layers aren't all the same shape either: the stack repeats a block of five sliding-window layers followed by one full-attention layer, five times over, the same 5-local-to-1-global pattern Google used in Gemma 3. A sliding-window layer can only look back across the most recent 1,024 tokens, however long the conversation gets; a full-attention layer can look all the way back to the first token. Both keep a real KV cache (this isn't the fixed-state linear-attention trick some other 2026 models use); the difference is how far back each layer is allowed to look.

Sliding-window attention1,024-token window, 8 KV heads × 256 dimFull attentionwhole context, 2 KV heads × 512 dim
layer 15:1 pattern × 5layer 30

5 of 30 layers keep a KV cache that grows with the full context. 25 more cap their cache at a fixed 1,024-token window, no matter how long the context gets.

Once a conversation passes 1,024 tokens, the 25 sliding-window layers stop growing: their cache stays capped at that window no matter how much further the context goes. Only the 5 full-attention layers keep accumulating, and Google gave them a narrower key/value width too (2 KV heads × 512 dim, versus 8 × 256 for the sliding-window layers), so even those five layers cost less per token than a same-sized full-attention layer normally would.

What a full 262,144-token context actually costs

Long context is usually where a local setup falls apart: the KV cache grows with every token fed to it, and on a conventional model it can overtake the weights long before the advertised window is full. Capping 25 of 30 layers at a 1,024-token window is what keeps that bill under control here.

0204060808k64k128k256k1,024-token window5.6 GBGemma 4 26B64.4 GBno split
Gemma 4 26B (MoE): 5 of 30 layers grow with contextThe same 26B if every layer used full, uncapped attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Filling the whole 262,144-token window costs 5.6 GB of KV cache instead of 64.4 GB. Add 15.3 GB of Q4_K_M weights and the full-context total is about 23.4 GB, right at the edge of a 24 GB card's real usable capacity (rated VRAM always leaves a little overhead), so it tips into a sliver of CPU offload there, while a 32 GB card holds it outright. The non-split version of the same model would have wanted 89.3 GB, squarely datacenter territory.

VRAM at each quantization

Calculated at 8k context. Because only part of this model's stack keeps a growing KV cache, longer sessions cost far less extra VRAM than a full-attention model of the same size.

QuantWeightsKV cacheTotal
FP32100.8 GB0.38 GB113.3 GB
BF1650.4 GB0.38 GB56.9 GB
FP1650.4 GB0.38 GB56.9 GB
Q8_026.8 GB0.38 GB30.4 GB
Q6_K20.7 GB0.38 GB23.6 GB
Q5_K_M17.9 GB0.38 GB20.5 GB
Q4_K_Mrec15.3 GB0.38 GB17.6 GB
Q3_K_M12.1 GB0.38 GB14.0 GB
Q2_K9.6 GB0.38 GB11.2 GB
NVFP4cuda12.6 GB0.38 GB14.5 GB

Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.

Quantization × context: where it fits

The table above holds context fixed at 8k. Add context as a second axis and the picture barely moves for this model: that flatness is the sliding-window cap paying off. Pick the row you can download, then the column you actually need.

Total VRAM needed by quantization and context length, with the GPU class each figure requires.
Quant8k ctx32k ctx128k ctx256k ctx
Q8_030.4 GB32 GB card31.0 GB32 GB card33.2 GB48 GB card36.3 GB48 GB card
Q6_K23.6 GB24 GB card24.2 GB32 GB card26.4 GB32 GB card29.4 GB32 GB card
Q5_K_M20.5 GB24 GB card21.1 GB24 GB card23.3 GB24 GB card26.3 GB32 GB card
Q4_K_Mrec17.6 GB24 GB card18.2 GB24 GB card20.4 GB24 GB card23.4 GB24 GB card
Q3_K_M14.0 GB16 GB card14.6 GB16 GB card16.8 GB24 GB card19.8 GB24 GB card
Each cell shows the smallest card that holds it:Consumer GPUWorkstation GPU

Benchmarks

MMLU-Pro
82.6
GPQA Diamond
82.3
LiveCodeBench
77.1

GPUs that run Gemma 4 26B (MoE) natively (91)

Show 86 more
Plus 14 GPUs that run it with CPU offload (slower)

Notes

MoE architecture (128 experts, 8 routed + 1 shared active per token) optimizing latency, activating ~3.8B parameters.

Hugging Face ↗Ollama ↗Released 2026-04-02

Compare Gemma 4 26B (MoE) with other models

How to run Gemma 4 26B (MoE) locally

816244880160320

Q4_K_M needs 17.6 GB: fits a single high-end consumer GPU (24 GB).

llama.cpp

./llama-server \
  -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M \
  -c 131072 -ngl 99 \
  --temp 1.0 --top-p 0.95 --top-k 64

Ollama

ollama run hf.co/unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M

LM Studio: Search for 'Gemma 4 26B' and take the UD-Q4_K_M build. Vision works out of the box, but audio is not supported on this variant; only the E2B/E4B/12B siblings ship an audio encoder.

Why this quantization? Q4_K_M puts the weights at roughly 15.3 GB, already past what a 16 GB card can hold once KV cache and overhead are added, so 20 GB+ is the practical floor. Because 25 of its 30 attention layers cap their KV cache at a 1,024-token window, context barely adds to that bill until the full 262,144-token window is close: a 24 GB card holds Q4_K_M comfortably out past 100k tokens, and only needs to spill into system RAM right at the very top of the window.

Who is Gemma 4 26B (MoE) for?

Owners of a single 20-24 GB GPU who want fast, general-purpose local inference with very long context headroom, without needing the full 26B loaded to feel slow; the MoE routing keeps decode closer to a dense ~4B model's speed.

Best for

  • Long-document summarization and multi-turn chat that stays well under the native 262k window
  • General-purpose assistant tasks where decode speed matters more than frontier-tier reasoning
  • Image understanding via the built-in vision encoder
  • Budget-conscious local setups where a 20-24 GB card is the whole GPU budget

Not ideal for

  • 16 GB GPUs, where even Q4_K_M weights alone leave little room for KV cache or overhead
  • Audio input, use the E2B, E4B, or 12B Gemma 4 siblings instead
  • Filling the entire 262,144-token window on a 24 GB card without any CPU offload
  • Anyone who needs the vendor-reported benchmark scores independently verified before relying on them

So should you run it?

On a single 24 GB card this is a comfortable pick for most conversations: Q4_K_M holds well past 100k of context with no offloading. Push all the way to the full 262,144-token window and even this model needs a sliver of CPU offload on 24 GB; step up to a 32 GB card if you routinely fill the window. The MoE routing means decode stays closer to a dense ~4B model's speed than a dense 26B's, even though the full 26B has to be resident in memory to make that routing fast.

Continue reading

vram-guides8 min
Best LLMs for 12 GB VRAM (2026)

Frequently asked questions

What are the VRAM requirements for Gemma 4 26B (MoE)?
Gemma 4 26B (MoE) requires approximately 17.6 GB of VRAM at Q4_K_M quantization, 30.4 GB at Q8, and 56.9 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Gemma 4 26B (MoE) have?
Gemma 4 26B (MoE) has 25.2 billion total parameters, but only 3.8 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
How capable is Gemma 4 26B (MoE)?
Gemma 4 26B (MoE) achieves an MMLU-Pro score of 82.6, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
Can Gemma 4 26B (MoE) run on a 16 GB GPU?
No. At Q4_K_M, Gemma 4 26B (MoE) needs 17.6 GB of VRAM, more than 16 GB. You will need a 24 GB GPU like the RTX 4090 or RTX 3090.
Can Gemma 4 26B (MoE) run on a 24 GB GPU?
Yes. Gemma 4 26B (MoE) fits in a 24 GB GPU at Q4_K_M, requiring 17.6 GB VRAM. GPUs with 24 GB include the RTX 4090, RTX 3090, and RTX 3090 Ti.
What is the smallest quantization for Gemma 4 26B (MoE) that fits in 24 GB of VRAM?
At NVFP4, Gemma 4 26B (MoE) needs 14.5 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Gemma 4 26B (MoE) locally?
A 24 GB GPU is the minimum. At Q4_K_M, Gemma 4 26B (MoE) needs 17.6 GB VRAM. Good options: RTX 4090 (24 GB), RTX 3090 (24 GB).