Nemotron 3 Super 120B
Nemotron 3 Super 120B needs roughly 96.5 GB VRAM at Q4_K_M quantization (268.9 GB at FP16). 24 GPUs we track can run it fully in VRAM at 8k context.
24 GPUs run this natively · 6 with CPU offload
Nemotron 3 Super 120B is a Mixture of Experts (MoE) model with 120B total parameters but only 12B active per token developed by NVIDIA. March 2026 120B MoE with 12B active. Hybrid Mamba-Transformer LatentMoE with 512 experts, 22 active.
To run Nemotron 3 Super 120B locally: Q4_K_M weights are 86.05GB (lmstudio-community's real GGUF; Ollama's default tag is about 87GB, and this site uses real file sizes for this model because its generic bits-per-weight estimate would say 73.1GB), about 96.5GB in total at a short context. That is past a 96GB RTX Pro 6000's real ceiling, so the smallest single GPUs this site tracks that hold it are the 128GB unified-memory machines (DGX Spark, M4 Max, M5 Max, Strix Halo), and an 80GB H100 or A100 holds Q3_K_M (67.23GB of weights, 75.4GB in total). Only 8 of its 88 layers keep a KV cache, so context is cheap: the full 1,048,576-token window adds 8.6GB (106.0GB in total at Q4_K_M), against 94.5GB if every layer cached. NVIDIA's own NVFP4 checkpoint is 80.3GB (the generic 4-bit estimate would be 60GB) and 90.0GB in total at 8k, which fits a 96GB RTX Pro 6000 or a DGX Spark; this site offers NVFP4 only on Blackwell cards. As a MoE model, inference speed depends on active parameters (12B) rather than total size.
Trained on 25T tokens, frontier agentic capabilities.
VRAM at each quantization
Nemotron 3 Super 120B natively supports a longer context window, but the table below is capped at 8k for comparability; its hybrid attention stack keeps KV cache growth well below linear.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 480.0 GB | 0.07 GB | 537.7 GB |
| BF16 | 240.0 GB | 0.07 GB | 268.9 GB |
| FP16 | 240.0 GB | 0.07 GB | 268.9 GB |
| Q8_0 | 127.6 GB | 0.07 GB | 142.9 GB |
| Q6_K | 112.9 GB | 0.07 GB | 126.5 GB |
| Q5_K_M | 95.7 GB | 0.07 GB | 107.3 GB |
| Q4_K_Mrec | 86.0 GB | 0.07 GB | 96.5 GB |
| Q3_K_M | 67.2 GB | 0.07 GB | 75.4 GB |
| Q2_K | 53.5 GB | 0.07 GB | 60.0 GB |
| NVFP4cuda | 80.3 GB | 0.07 GB | 90.0 GB |
Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.
Benchmarks
GPUs that run Nemotron 3 Super 120B natively (24)
- NVIDIA B300 288GBBF16 · 50.3 t/s
- NVIDIA B200 180GBNVFP4 · 87.1 t/s
- NVIDIA H200 141GBQ6_K · 70.1 t/s
- NVIDIA H100 80GBQ3_K_M · 82.4 t/s
- NVIDIA A100 80GBQ3_K_M · 73.5 t/s
Show 19 more
- NVIDIA RTX Pro 6000NVFP4 · 81 t/s
- NVIDIA DGX Spark (128GB)NVFP4 · 24.4 t/s
- AMD Instinct MI300XQ8_0 · 42.5 t/s
- AMD Strix Halo (128GB)Q5_K_M · 17.1 t/s
- AMD Strix Halo (96GB)Q3_K_M · 21.7 t/s
- Apple M5 Ultra (512GB)BF16 · 22.5 t/s
- Apple M5 Ultra (256GB)Q8_0 · 30.6 t/s
- Apple M5 Ultra (96GB)Q3_K_M · 37.8 t/s
- Apple M5 Max (128GB)Q5_K_M · 25.7 t/s
- Apple M4 Max (128GB)Q5_K_M · 24.1 t/s
- Apple M3 Ultra (512GB)BF16 · 17.8 t/s
- Apple M3 Ultra (256GB)Q8_0 · 25.7 t/s
- Apple M3 Ultra (96GB)Q3_K_M · 33.6 t/s
- Apple M3 Max (128GB)Q5_K_M · 20.2 t/s
- Apple M3 Max (96GB)Q3_K_M · 21 t/s
- Apple M2 Ultra (192GB)Q8_0 · 25.4 t/s
- Apple M2 Max (96GB)Q3_K_M · 24.6 t/s
- Apple M1 Ultra (128GB)Q5_K_M · 29.1 t/s
- Intel Data Center GPU Max 1550Q5_K_M · 43.7 t/s
Plus 6 GPUs that run it with CPU offload (slower)
- NVIDIA A100 40GBQ2_K · 18.7 t/s
- NVIDIA L40SQ2_K · 23.6 t/s
- NVIDIA RTX A6000Q2_K · 24.2 t/s
- NVIDIA RTX 6000 AdaQ2_K · 25.1 t/s
- AMD Radeon PRO W7900Q2_K · 20.4 t/s
- Intel Data Center GPU Max 1100Q2_K · 20.4 t/s
Cloud GPU Rental
Cheapest way to run Nemotron 3 Super 120B: 1× RTX Pro 6000 at NVFP4, ~81 tok/s, from $1.69/hr.
Pay by the hour · no contract · pods start in about a minute.
Rent this setup on RunPod ↗ (+$5 signup credit)Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing.
Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.
Notes
Hybrid Mamba-Transformer LatentMoE with 512 routed experts, 22 active per token. Only 8 of its 88 layers keep a KV cache that grows with context. Trained on 25T tokens.
Continue reading
Frequently asked questions
- What are the VRAM requirements for Nemotron 3 Super 120B?
- Nemotron 3 Super 120B requires approximately 96.5 GB of VRAM at Q4_K_M quantization, 142.9 GB at Q8, and 268.9 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
- How many parameters does Nemotron 3 Super 120B have?
- Nemotron 3 Super 120B has 120 billion total parameters, but only 12 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is Nemotron 3 Super 120B?
- Nemotron 3 Super 120B achieves an MMLU-Pro score of 83.7, placing it among the most capable open-weight models available, competitive with frontier systems on general knowledge and reasoning.
- Can Nemotron 3 Super 120B run on a 16 GB GPU?
- No. At Q4_K_M, Nemotron 3 Super 120B needs 96.5 GB of VRAM, more than 16 GB. You will need a multi-GPU server.
- Can Nemotron 3 Super 120B run on a 24 GB GPU?
- No. Even at Q4_K_M, Nemotron 3 Super 120B needs 96.5 GB. Consider a multi-GPU server with 97 GB+ of combined VRAM.
- What is the smallest quantization for Nemotron 3 Super 120B that fits in 24 GB of VRAM?
- Nemotron 3 Super 120B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 60.0 GB at Q2_K.
- What GPU do I need to run Nemotron 3 Super 120B locally?
- You need a multi-GPU server. At Q4_K_M, Nemotron 3 Super 120B needs 96.5 GB VRAM, more than any single consumer GPU. That's roughly 2x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.
- Can I run Nemotron 3 Super 120B in the cloud?
- Yes. The cheapest RunPod setup we found that holds Nemotron 3 Super 120B entirely in GPU memory at 8k context is 1× RTX Pro 6000 at NVFP4, at roughly 81 tokens per second and from $1.69 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.