Nemotron 3.5 Lightning 30B-A3B vs Gemma 4 26B (MoE)

Analysis

Nemotron 3.5 Lightning 30B-A3B and Gemma 4 26B are both MoE models branded around the same 26-30B range (Nemotron's real total is 30B, Gemma's own model card lists 25.2B despite the "26B" name) and released four-plus months apart in 2026 (Gemma on 2 April, Nemotron on 11 August). Each hides a different second trick behind its MoE routing: Gemma keeps every layer a real attention layer but caps most of them to a short sliding window; Nemotron swaps most of its layers for Mamba-2 state-space blocks that skip attention, and a KV cache, entirely. The two land at close to the same VRAM total within the context range they share, for structurally different reasons.

Nemotron routes through 128 experts (6 active plus 1 shared per token, 3B of 30B parameters active), and only 6 of its 52 layers keep any KV cache at all (2 KV heads x 128 head dim, 256-wide); the rest split between 26 Mamba-2 layers and 20 MoE FFN layers with nothing to cache. Gemma also routes through 128 experts (8 active plus 1 shared, 3.8B of 25.2B active), but all 30 of its layers are real attention: 25 of them cap at a 1,024-token sliding window (8 KV heads x 256 head dim, 2,048-wide while active), and the remaining 5 full-attention layers use a narrower 2 KV heads x 512 head dim, 1,024-wide, Google's own inversion of the usual pattern. Both models natively reach 262,144 tokens (Nemotron's window continues further, to 1,048,576). At that shared ceiling, Nemotron's KV cache is 1.6 GB against Gemma's 5.6 GB, still smaller despite Gemma's window cap, because Nemotron only ever accumulates a cache on 6 of 52 layers (about a ninth) against Gemma's full 30 of 30 (capped, not skipped, on 25 of them). Add each model's Q4_K_M weights and the full picture is 22.3 GB total for Nemotron against 23.4 GB for Gemma, close enough that the difference matters at the edge of a 24 GB card: Nemotron's total clears the card's real ~22.8 GB usable ceiling, Gemma's doesn't, and tips into a sliver of CPU offload at that same context length. The two share only two benchmark fields: MMLU-Pro, where Gemma edges ahead (82.6 vs 81.62), and GPQA Diamond, where Gemma leads more clearly (82.3 vs 75.57). Beyond that they report different suites entirely, Nemotron's own SWE-bench Verified (52.80) against Gemma's own LiveCodeBench v6 (77.1), so those two numbers aren't a head-to-head. NVIDIA isn't chasing benchmark leadership with Lightning's smaller MoE footprint (3B active): the pitch is throughput, up to 4x higher output and 30% faster task completion against similarly-sized open models by NVIDIA's own figures. Both ship under Apache-family permissive licenses (Nemotron under NVIDIA's own OpenMDW-1.1, Gemma under standard Apache 2.0); Gemma additionally accepts image input where Nemotron is text-only. Tooling clearly favors Gemma at this point: four-plus months after launch it already has Ollama tags and Unsloth GGUF quants, while Nemotron shipped 11 August 2026 with no GGUF or Ollama build at all, native NVFP4, FP8, and BF16 checkpoints for vLLM, SGLang, and TensorRT-LLM only.

Bottom line: Choose Nemotron 3.5 Lightning if the priority is squeezing the most out of a 24 GB card at long context, or raw agentic throughput, since NVIDIA's own throughput claims are exactly what its smaller MoE footprint buys. Gemma 4 26B is the safer default almost everywhere else: it scores higher on both shared academic benchmarks, accepts image input Nemotron doesn't, and already has the mature GGUF and Ollama support that Nemotron's day-one release is still missing.