Nemotron 3 Super 120B
Nemotron 3 Super 120B needs roughly 82.7 GB VRAM at Q4_K_M quantization (269.6 GB at FP16). 29 GPUs we track can run it fully in VRAM at 8k context.
29 GPUs run this natively · 10 with CPU offload
Nemotron 3 Super 120B is a Mixture of Experts (MoE) model with 120B total parameters but only 12B active per token developed by NVIDIA. March 2026 120B MoE with 12B active. Hybrid Mamba-Transformer LatentMoE with 512 experts, 22 active.
To run Nemotron 3 Super 120B locally: Q3_K ~50-60GB — requires 80GB GPU or dual 48GB. As a MoE model, inference speed depends on active parameters (12B) rather than total size.
Trained on 25T tokens — frontier agentic capabilities.
VRAM at each quantization
Nemotron 3 Super 120B natively supports a longer context window, but the table below is capped at 8k for comparability — KV cache grows linearly with context length.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 480.0 GB | 0.74 GB | 538.4 GB |
| BF16 | 240.0 GB | 0.74 GB | 269.6 GB |
| FP16 | 240.0 GB | 0.74 GB | 269.6 GB |
| Q8_0 | 127.6 GB | 0.74 GB | 143.7 GB |
| Q6_K | 98.5 GB | 0.74 GB | 111.2 GB |
| Q5_K_M | 85.4 GB | 0.74 GB | 96.5 GB |
| Q4_K_M | 73.1 GB | 0.74 GB | 82.7 GB |
| Q3_K_Mrec | 57.7 GB | 0.74 GB | 65.5 GB |
| Q2_K | 45.7 GB | 0.74 GB | 52.0 GB |
| NVFP4cuda | 60.0 GB | 0.74 GB | 68.0 GB |
Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.
Benchmarks
GPUs that run Nemotron 3 Super 120B natively (29)
- NVIDIA H100 80GBNVFP4 · 105 t/s
- NVIDIA A100 80GBNVFP4 · 63.9 t/s
- NVIDIA RTX Pro 6000NVFP4 · 42.1 t/s
- NVIDIA DGX Spark (128GB)NVFP4 · 8.6 t/s
- AMD Instinct MI300XQ8_0 · 79.6 t/s
- AMD Strix Halo (128GB)Q6_K · 5 t/s
- AMD Strix Halo (96GB)Q4_K_M · 6.6 t/s
- AMD Strix Halo (64GB)Q2_K · 10.4 t/s
- Apple M5 Max (128GB)Q6_K · 14.6 t/s
- Apple M5 Max (64GB)Q2_K · 30.7 t/s
- Apple M4 Ultra (384GB)BF16 · 10.8 t/s
- Apple M4 Ultra (192GB)Q8_0 · 20.2 t/s
- Apple M4 Max (128GB)Q6_K · 13 t/s
- Apple M4 Max (96GB)Q4_K_M · 17.4 t/s
- Apple M4 Max (64GB)Q2_K · 27.3 t/s
- Apple M3 Ultra (512GB)BF16 · 8.1 t/s
- Apple M3 Ultra (256GB)Q8_0 · 15.1 t/s
- Apple M3 Ultra (96GB)Q4_K_M · 26.1 t/s
- Apple M3 Max (128GB)Q6_K · 9.5 t/s
- Apple M3 Max (96GB)Q4_K_M · 12.7 t/s
- Apple M3 Max (64GB)Q2_K · 20 t/s
- Apple M2 Ultra (384GB)BF16 · 7.9 t/s
- Apple M2 Ultra (192GB)Q8_0 · 14.8 t/s
- Apple M2 Max (96GB)Q4_K_M · 12.7 t/s
- Apple M2 Max (64GB)Q2_K · 20 t/s
- Apple M1 Ultra (128GB)Q6_K · 19.1 t/s
- Apple M1 Ultra (64GB)Q2_K · 40.1 t/s
- Apple M1 Max (64GB)Q2_K · 20 t/s
- Intel Data Center GPU Max 1550Q6_K · 63.4 t/s
Plus 10 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 5090Q2_K · 4.3 t/s
- NVIDIA A100 40GBQ2_K · 7.7 t/s
- NVIDIA L40SNVFP4 · 4.3 t/s
- NVIDIA RTX A6000NVFP4 · 4.2 t/s
- NVIDIA RTX 5000 AdaQ2_K · 4 t/s
- NVIDIA RTX 6000 AdaNVFP4 · 4.3 t/s
- AMD Radeon PRO W7800Q2_K · 4 t/s
- AMD Radeon PRO W7900Q3_K_M · 4.9 t/s
- AMD Radeon AI Pro 9700 32GBQ2_K · 4.1 t/s
- Intel Data Center GPU Max 1100Q3_K_M · 5.1 t/s
Notes
Hybrid Mamba-Transformer LatentMoE with 512 routed experts, 22 active per token. Trained on 25T tokens.
Frequently asked questions
- What are the VRAM requirements for Nemotron 3 Super 120B?
- Nemotron 3 Super 120B requires approximately 82.7 GB of VRAM at Q4_K_M quantization, 143.7 GB at Q8, and 269.6 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Nemotron 3 Super 120B have?
- Nemotron 3 Super 120B has 120 billion total parameters, but only 12 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is Nemotron 3 Super 120B?
- Nemotron 3 Super 120B achieves an MMLU-Pro score of 83.7, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
- Can Nemotron 3 Super 120B run on a 16 GB GPU?
- No. At Q4_K_M, Nemotron 3 Super 120B needs 82.7 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Nemotron 3 Super 120B run on a 24 GB GPU?
- No. Even at Q4_K_M, Nemotron 3 Super 120B needs 82.7 GB. Consider a multi-GPU server with 80 GB+ total VRAM.
- What is the smallest quantization for Nemotron 3 Super 120B that fits in 24 GB of VRAM?
- Nemotron 3 Super 120B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 52.0 GB at Q2_K.
- What GPU do I need to run Nemotron 3 Super 120B locally?
- You need a multi-GPU server. At Q4_K_M, Nemotron 3 Super 120B needs 82.7 GB VRAM, more than any single consumer GPU. Consider 2–4× H100 or A100 GPUs.