Nemotron 3 Super 120B
Nemotron 3 Super 120B needs roughly 76.5 GB VRAM at Q4_K_M quantization (269.6 GB at FP16). 34 GPUs we track can run it fully in VRAM at 8k context.
34 GPUs run this natively · 12 with CPU offload
Nemotron 3 Super 120B is a Mixture of Experts (MoE) model with 120B total parameters but only 12B active per token developed by NVIDIA. March 2026 120B MoE with 12B active. Hybrid Mamba-Transformer LatentMoE with 512 experts, 22 active.
To run Nemotron 3 Super 120B locally: Q3_K ~50-60GB — requires 80GB GPU or dual 48GB. As a MoE model, inference speed depends on active parameters (12B) rather than total size.
Trained on 25T tokens — frontier agentic capabilities.
VRAM at each quantization
Assumes 8k context. KV cache grows linearly with context length.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 480.0 GB | 0.74 GB | 538.4 GB |
| BF16 | 240.0 GB | 0.74 GB | 269.6 GB |
| FP16 | 240.0 GB | 0.74 GB | 269.6 GB |
| Q8_0 | 120.0 GB | 0.74 GB | 135.2 GB |
| Q6_K | 98.4 GB | 0.74 GB | 111.0 GB |
| Q5_K_M | 77.3 GB | 0.74 GB | 87.4 GB |
| Q4_K_M | 67.6 GB | 0.74 GB | 76.5 GB |
| Q3_K_Mrec | 51.6 GB | 0.74 GB | 58.6 GB |
| Q2_K | 39.5 GB | 0.74 GB | 45.0 GB |
| NVFP4cuda | 60.0 GB | 0.74 GB | 68.0 GB |
KV cache shown at 8k context (FP16). NVFP4 requires a CUDA GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Nemotron 3 Super 120B natively (34)
- NVIDIA H100 80GBNVFP4 · 614.2 t/s
- NVIDIA A100 80GBNVFP4 · 373.8 t/s
- NVIDIA L40SQ2_K · 240.7 t/s
- NVIDIA RTX A6000Q2_K · 214 t/s
- NVIDIA RTX 6000 AdaQ2_K · 267.5 t/s
- NVIDIA RTX Pro 6000NVFP4 · 246.4 t/s
- NVIDIA DGX Spark (128GB)NVFP4 · 50.1 t/s
- AMD Radeon PRO W7900Q2_K · 240.7 t/s
- AMD Instinct MI300XQ8_0 · 485.8 t/s
- AMD Strix Halo (128GB)Q6_K · 28.6 t/s
- AMD Strix Halo (96GB)Q5_K_M · 36.4 t/s
- AMD Strix Halo (64GB)Q3_K_M · 54.6 t/s
- Apple M5 Max (128GB)Q6_K · 68.6 t/s
- Apple M5 Max (64GB)Q3_K_M · 130.9 t/s
- Apple M4 Ultra (384GB)BF16 · 50.1 t/s
- Apple M4 Ultra (192GB)Q8_0 · 100.1 t/s
- Apple M4 Max (128GB)Q6_K · 61 t/s
- Apple M4 Max (96GB)Q5_K_M · 77.7 t/s
- Apple M4 Max (64GB)Q3_K_M · 116.4 t/s
- Apple M3 Ultra (512GB)BF16 · 37.5 t/s
- Apple M3 Ultra (256GB)Q8_0 · 75.1 t/s
- Apple M3 Ultra (96GB)Q5_K_M · 116.6 t/s
- Apple M3 Max (128GB)Q6_K · 44.7 t/s
- Apple M3 Max (96GB)Q5_K_M · 56.9 t/s
- Apple M3 Max (64GB)Q3_K_M · 85.3 t/s
- Apple M2 Ultra (384GB)BF16 · 36.7 t/s
- Apple M2 Ultra (192GB)Q8_0 · 73.3 t/s
- Apple M2 Max (96GB)Q5_K_M · 56.9 t/s
- Apple M2 Max (64GB)Q3_K_M · 85.3 t/s
- Apple M1 Ultra (128GB)Q6_K · 89.4 t/s
- Apple M1 Ultra (64GB)Q3_K_M · 170.5 t/s
- Apple M1 Max (64GB)Q3_K_M · 85.3 t/s
- Intel Data Center GPU Max 1550Q6_K · 366.2 t/s
- Intel Data Center GPU Max 1100Q2_K · 342.4 t/s
Plus 12 GPUs that run it with CPU offload (slower)
- NVIDIA RTX 5090Q2_K · 113.5 t/s
- NVIDIA RTX 4090Q2_K · 63.8 t/s
- NVIDIA RTX 3090Q2_K · 59.3 t/s
- NVIDIA RTX 3090 TiQ2_K · 63.8 t/s
- NVIDIA A100 40GBQ3_K_M · 75.3 t/s
- NVIDIA RTX 4500 AdaQ2_K · 27.4 t/s
- NVIDIA RTX 5000 AdaQ2_K · 36.5 t/s
- AMD Radeon RX 7900 XTXQ2_K · 60.8 t/s
- AMD Radeon PRO W7800Q2_K · 36.5 t/s
- AMD Radeon AI Pro 9700 32GBQ2_K · 40.5 t/s
- Intel Arc Pro B70 24GBQ2_K · 28.9 t/s
- Intel Arc Pro B60 24GBQ2_K · 24.1 t/s
Notes
Hybrid Mamba-Transformer LatentMoE with 512 routed experts, 22 active per token. Trained on 25T tokens.
Frequently asked questions
- What are the VRAM requirements for Nemotron 3 Super 120B?
- Nemotron 3 Super 120B requires approximately 76.5 GB of VRAM at Q4_K_M quantization, 135.2 GB at Q8, and 269.6 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Nemotron 3 Super 120B have?
- Nemotron 3 Super 120B has 120 billion total parameters, but only 12 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is Nemotron 3 Super 120B?
- Nemotron 3 Super 120B achieves an MMLU-Pro score of 83.7, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
- Can Nemotron 3 Super 120B run on a 16 GB GPU?
- No. At Q4_K_M, Nemotron 3 Super 120B needs 76.5 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Nemotron 3 Super 120B run on a 24 GB GPU?
- No. Even at Q4_K_M, Nemotron 3 Super 120B needs 76.5 GB. Consider a multi-GPU server with 80 GB+ total VRAM.
- What is the smallest quantization for Nemotron 3 Super 120B that fits in 24 GB of VRAM?
- Nemotron 3 Super 120B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 45.0 GB at Q2_K.
- What GPU do I need to run Nemotron 3 Super 120B locally?
- You need a multi-GPU server. At Q4_K_M, Nemotron 3 Super 120B needs 76.5 GB VRAM, more than any single consumer GPU. Consider 2–4× H100 or A100 GPUs.