Nemotron 3 Ultra 550B-A55B
Nemotron 3 Ultra 550B-A55B needs roughly 376.2 GB VRAM at Q4_K_M quantization (1233.0 GB at FP16). 4 GPUs we track can run it fully in VRAM at 8k context.
4 GPUs run this natively · 0 with CPU offload
Nemotron 3 Ultra 550B-A55B is a Mixture of Experts (MoE) large language model with 550B total parameters but only 55B active per token developed by NVIDIA. Released in June 2026, it is a text-only model with a 1M context window, released under the OpenMDW 1.1 license, allowing commercial use. Datacenter-scale hybrid Mamba-attention LatentMoE with 512 routed experts, 22 selected experts plus a shared expert, and multi-token prediction. NVIDIA supports up to 1M context; the native config is 256K before long-context scaling. NVIDIA reports MMLU-Pro 86.8, GPQA-Diamond 87.0, SWE-bench Verified 70.7, and Terminal-Bench 2.1 56.4 for the BF16 model. The official NVFP4 checkpoint needs roughly 4x B200/GB200-class GPUs or 8x H100s, so a single consumer GPU is not a realistic target.
To run Nemotron 3 Ultra 550B-A55B locally, you need approximately 376.2 GB of VRAM at Q4_K_M quantization with 8k context. 4 of the GPUs we track can run it fully in VRAM, with a further 0 able to offload to system RAM. At Q4_K_M it requires 376.2 GB — more than any single consumer GPU. A multi-GPU server or 80 GB datacenter GPU is required. At Q8_K_M (655.8 GB), you get near-FP16 quality while still fitting on large server or multi-GPU setups. FP16 requires 1233.0 GB, limiting it to datacenter-class hardware with 80 GB+ VRAM. As a MoE model, inference speed depends on active parameters (55B) rather than total size, so once it fits in VRAM it runs noticeably faster than dense models of the same parameter count.
Its MMLU-Pro score of 86.8 places it among the strongest open-weight models available. The license allows commercial use.
VRAM at each quantization
Calculated at 8k context. Since KV cache scales linearly with context, longer sessions need more VRAM than shown here.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 2200.0 GB | 0.91 GB | 2465.0 GB |
| BF16 | 1100.0 GB | 0.91 GB | 1233.0 GB |
| FP16 | 1100.0 GB | 0.91 GB | 1233.0 GB |
| Q8_0 | 584.6 GB | 0.91 GB | 655.8 GB |
| Q6_K | 451.6 GB | 0.91 GB | 506.8 GB |
| Q5_K_M | 391.6 GB | 0.91 GB | 439.6 GB |
| Q4_K_M | 334.9 GB | 0.91 GB | 376.2 GB |
| Q3_K_M | 264.6 GB | 0.91 GB | 297.3 GB |
| Q2_K | 209.6 GB | 0.91 GB | 235.7 GB |
| NVFP4reccuda | 275.0 GB | 0.91 GB | 309.0 GB |
KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Nemotron 3 Ultra 550B-A55B natively (4)
- Apple M4 Ultra (384GB)Q3_K_M · 9.8 t/s
- Apple M3 Ultra (512GB)Q5_K_M · 5 t/s
- Apple M3 Ultra (256GB)Q2_K · 9.3 t/s
- Apple M2 Ultra (384GB)Q3_K_M · 7.2 t/s
Notes
Datacenter-scale hybrid Mamba-attention LatentMoE with 512 routed experts, 22 selected experts plus a shared expert, and multi-token prediction. NVIDIA supports up to 1M context; the native config is 256K before long-context scaling. NVIDIA reports MMLU-Pro 86.8, GPQA-Diamond 87.0, SWE-bench Verified 70.7, and Terminal-Bench 2.1 56.4 for the BF16 model. The official NVFP4 checkpoint needs roughly 4x B200/GB200-class GPUs or 8x H100s, so a single consumer GPU is not a realistic target.
How to run Nemotron 3 Ultra 550B-A55B locally
NVFP4 needs 309.0 GB — needs multiple datacenter-class GPUs (80 GB+ each).
vLLM (official NVFP4 checkpoint)
vllm serve nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4LM Studio: Not a desktop-class model. Even NVIDIA's NVFP4 checkpoint needs roughly four B200/GB200-class GPUs or eight H100s; use a distributed vLLM or SGLang deployment.
Why this quantization? NVFP4 is NVIDIA's supported low-bit deployment path and is a much better representation of this model than an invented consumer GGUF recommendation. It reduces the 550B weight pool to roughly 275 GB before runtime overhead, but the official minimum remains a multi-GPU datacenter system.
Who is Nemotron 3 Ultra 550B-A55B for?
Infrastructure teams and researchers with Blackwell- or H100-class multi-GPU servers who need an open, long-context reasoning and coding model with NVIDIA's optimized inference stack.
Best for
- Long-running coding and tool-use agents
- Million-token document or repository workloads
- High-throughput self-hosted reasoning services
Not ideal for
- Consumer GPUs or workstations
- CPU offload across ordinary system RAM
- Users expecting an Ollama or LM Studio desktop download
Frequently asked questions
- What are the VRAM requirements for Nemotron 3 Ultra 550B-A55B?
- Nemotron 3 Ultra 550B-A55B requires approximately 376.2 GB of VRAM at Q4_K_M quantization, 655.8 GB at Q8, and 1233.0 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Nemotron 3 Ultra 550B-A55B have?
- Nemotron 3 Ultra 550B-A55B has 550 billion total parameters, but only 55 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- How capable is Nemotron 3 Ultra 550B-A55B?
- Nemotron 3 Ultra 550B-A55B achieves an MMLU-Pro score of 86.8, placing it among the most capable open-weight models available — competitive with frontier systems on general knowledge and reasoning.
- Can Nemotron 3 Ultra 550B-A55B run on a 16 GB GPU?
- No. At Q4_K_M, Nemotron 3 Ultra 550B-A55B needs 376.2 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Nemotron 3 Ultra 550B-A55B run on a 24 GB GPU?
- No. Even at Q4_K_M, Nemotron 3 Ultra 550B-A55B needs 376.2 GB. Consider a multi-GPU server with 80 GB+ total VRAM.
- What is the smallest quantization for Nemotron 3 Ultra 550B-A55B that fits in 24 GB of VRAM?
- Nemotron 3 Ultra 550B-A55B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 235.7 GB at Q2_K.
- What GPU do I need to run Nemotron 3 Ultra 550B-A55B locally?
- You need a multi-GPU server. At Q4_K_M, Nemotron 3 Ultra 550B-A55B needs 376.2 GB VRAM, more than any single consumer GPU. Consider 2–4× H100 or A100 GPUs.