CanItRun Logocanitrun.

Nemotron 3 Nano 30B

Nemotron 3 Nano 30B needs roughly 27.5 GB VRAM at Q4_K_M quantization (71.7 GB at FP16). 68 GPUs we track can run it fully in VRAM at 8k context.

68 GPUs run this natively · 35 with CPU offload

NVIDIA32B params3B active (MoE)1024k contextNVIDIACommercial use ok

Nemotron 3 Nano 30B is a Mixture of Experts (MoE) model with 32B total parameters but only 3B active per token developed by NVIDIA. December 2025 32B MoE with only 3B active per token. Hybrid Mamba-Transformer architecture with 1M context.

To run Nemotron 3 Nano 30B locally: Q4_K_M weights are 24.52GB (lmstudio-community's real GGUF; Ollama's default tag is 24GB, and this site uses real file sizes for this model because its generic bits-per-weight estimate would say 19.5GB), about 27.5GB in total at 8K context. That is past a 24GB card, where Q3_K_M (19.85GB of weights, about 22.3GB in total) is the largest build that fits natively; a 32GB RTX 5090 holds Q5_K_M (26.04GB, 29.2GB in total). Only 6 of its 52 layers keep a KV cache, so long context stays cheap: the full 1,048,576-token window adds 6.4GB. NVIDIA's own NVFP4 checkpoint is 19.34GB (the generic 4-bit estimate would be 16.0GB), 21.7GB in total at 8K, on Blackwell cards only. Exceptional tokens/sec due to 3B active params. As a MoE model, inference speed depends on active parameters (3B) rather than total size.

Optimized for agentic workflows: MoE efficiency with Mamba speed.

VRAM at each quantization

Calculated at 8k context. Because only part of this model's stack keeps a growing KV cache, longer sessions cost far less extra VRAM than a full-attention model of the same size.

QuantWeightsKV cacheTotal
FP32128.0 GB0.05 GB143.4 GB
BF1664.0 GB0.05 GB71.7 GB
FP1664.0 GB0.05 GB71.7 GB
Q8_034.0 GB0.05 GB38.1 GB
Q6_K33.5 GB0.05 GB37.6 GB
Q5_K_Mrec26.0 GB0.05 GB29.2 GB
Q4_K_M24.5 GB0.05 GB27.5 GB
Q3_K_M19.9 GB0.05 GB22.3 GB
Q2_K17.9 GB0.05 GB20.1 GB
NVFP4cuda19.3 GB0.05 GB21.7 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Benchmarks

MMLU-Pro
78.3

GPUs that run Nemotron 3 Nano 30B natively (68)

Show 63 more
Plus 35 GPUs that run it with CPU offload (slower)

No GPU? Rent an RTX A6000 from $0.33/hr ↗

Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

Hybrid Mamba-Transformer MoE: 30B total / 3B active per token. Optimized for agentic workflows.

Hugging Face ↗Ollama ↗Released 2025-12-01

Continue reading

vram-guides10 min
Best LLMs for 12 GB VRAM (2026): RTX 3060, 4070 & 5070
vram-guides9 min
Best LLMs for 24 GB VRAM (2026)

Frequently asked questions

What are the VRAM requirements for Nemotron 3 Nano 30B?
Nemotron 3 Nano 30B requires approximately 27.5 GB of VRAM at Q4_K_M quantization, 38.1 GB at Q8, and 71.7 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Nemotron 3 Nano 30B have?
Nemotron 3 Nano 30B has 32 billion total parameters, but only 3 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
How capable is Nemotron 3 Nano 30B?
Nemotron 3 Nano 30B achieves an MMLU-Pro score of 78.3, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
Can Nemotron 3 Nano 30B run on a 16 GB GPU?
No. At Q4_K_M, Nemotron 3 Nano 30B needs 27.5 GB of VRAM, more than 16 GB. You will need a 32 GB GPU like the RTX 5090.
Can Nemotron 3 Nano 30B run on a 24 GB GPU?
No. Even at Q4_K_M, Nemotron 3 Nano 30B needs 27.5 GB. Consider a 32 GB card like the RTX 5090.
What is the smallest quantization for Nemotron 3 Nano 30B that fits in 24 GB of VRAM?
At NVFP4, Nemotron 3 Nano 30B needs 21.7 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Nemotron 3 Nano 30B locally?
A 32 GB GPU is the minimum. At Q4_K_M, Nemotron 3 Nano 30B needs 27.5 GB VRAM. Good option: RTX 5090 (32 GB).
Can I run Nemotron 3 Nano 30B in the cloud?
Yes. The cheapest RunPod setup we found that holds Nemotron 3 Nano 30B entirely in GPU memory at 8k context is 1× RTX A6000 at Q8_0, at roughly 110.4 tokens per second and from $0.33 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.