Nemotron 3.5 Lightning 30B-A3B vs Gemma 4 26B (MoE)

Analysis

Nemotron 3.5 Lightning 30B-A3B and Gemma 4 26B are both MoE models branded around the same 26-30B range (Nemotron's real total is 30B, Gemma's own model card lists 25.2B despite the "26B" name) and released four-plus months apart in 2026 (Gemma on 2 April, Nemotron on 11 August). Each hides a different second trick behind its MoE routing: Gemma keeps every layer a real attention layer but caps most of them to a short sliding window; Nemotron swaps most of its layers for Mamba-2 state-space blocks that skip attention, and a KV cache, entirely. The two land at close to the same VRAM total within the context range they share, for structurally different reasons.

Nemotron routes through 128 experts (6 active plus 1 shared per token, 3B of 30B parameters active), and only 6 of its 52 layers keep any KV cache at all (2 KV heads x 128 head dim, 256-wide); the rest split between 26 Mamba-2 layers and 20 MoE FFN layers with nothing to cache. Gemma also routes through 128 experts (8 active plus 1 shared, 3.8B of 25.2B active), but all 30 of its layers are real attention: 25 of them cap at a 1,024-token sliding window (8 KV heads x 256 head dim, 2,048-wide while active), and the remaining 5 full-attention layers use a narrower 2 KV heads x 512 head dim, 1,024-wide, Google's own inversion of the usual pattern. Both models natively reach 262,144 tokens (Nemotron's window continues further, to 1,048,576). At that shared ceiling, Nemotron's KV cache is 1.6 GB against Gemma's 5.6 GB, still smaller despite Gemma's window cap, because Nemotron only ever accumulates a cache on 6 of 52 layers (about a ninth) against Gemma's full 30 of 30 (capped, not skipped, on 25 of them). Add each model's Q4_K_M weights and the full picture is 30.3 GB total for Nemotron against 23.4 GB for Gemma, a gap that decides the card: Gemma's total sits just past a 24 GB card's real ~22.8 GB usable ceiling and tips into a sliver of CPU offload at that context length, while Nemotron needs a 32 GB card (about 30.4 GB usable). Nemotron's real Q4_K_M file is 25.5 GB, so its smaller MoE footprint saves KV cache, not weights. The two share only two benchmark fields: MMLU-Pro, where Gemma edges ahead (82.6 vs 81.62), and GPQA Diamond, where Gemma leads more clearly (82.3 vs 75.57). Beyond that they report different suites entirely, Nemotron's own SWE-bench Verified (52.80) against Gemma's own LiveCodeBench v6 (77.1), so those two numbers aren't a head-to-head. NVIDIA isn't chasing benchmark leadership with Lightning's smaller MoE footprint (3B active): the pitch is throughput, up to 4x higher output and 30% faster task completion against similarly-sized open models by NVIDIA's own figures. Both ship under Apache-family permissive licenses (Nemotron under NVIDIA's own OpenMDW-1.1, Gemma under standard Apache 2.0); Gemma additionally accepts image input where Nemotron is text-only. Tooling still favors Gemma: four-plus months after launch it has Ollama tags and Unsloth GGUF quants, while Nemotron shipped 11 August 2026 with only native NVFP4, FP8, and BF16 checkpoints for vLLM, SGLang, and TensorRT-LLM and gained community GGUFs and an Ollama entry afterward.

Bottom line: Choose Nemotron 3.5 Lightning if the priority is raw agentic throughput or a context window past Gemma's, since NVIDIA's own throughput claims are what its 3B-active design buys; it costs more memory than Gemma, though (30.3 GB against 23.4 GB at 262,144 tokens). Gemma 4 26B is the safer default almost everywhere else: it scores higher on both shared academic benchmarks, accepts image input Nemotron doesn't, and has had longer to build up the GGUF and Ollama support that Nemotron only picked up after its day-one release.