What are the VRAM requirements for Nemotron 3.5 Super VL 120B-A12B?

Nemotron 3.5 Super VL 120B-A12B requires approximately 81.9 GB of VRAM at Q4_K_M quantization, 142.9 GB at Q8, and 268.9 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.

How many parameters does Nemotron 3.5 Super VL 120B-A12B have?

Nemotron 3.5 Super VL 120B-A12B has 120 billion total parameters, but only 12 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.

Is Nemotron 3.5 Super VL 120B-A12B good for coding?

Yes. Nemotron 3.5 Super VL 120B-A12B scores 88.2 on LiveCodeBench, demonstrating strong code generation and completion capabilities.

Can Nemotron 3.5 Super VL 120B-A12B run on a 16 GB GPU?

No. At Q4_K_M, Nemotron 3.5 Super VL 120B-A12B needs 81.9 GB of VRAM, more than 16 GB. You will need a multi-GPU server.

Can Nemotron 3.5 Super VL 120B-A12B run on a 24 GB GPU?

No. Even at Q4_K_M, Nemotron 3.5 Super VL 120B-A12B needs 81.9 GB. Consider a multi-GPU server with 82 GB+ of combined VRAM.

What is the smallest quantization for Nemotron 3.5 Super VL 120B-A12B that fits in 24 GB of VRAM?

Nemotron 3.5 Super VL 120B-A12B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 51.3 GB at Q2_K.

What GPU do I need to run Nemotron 3.5 Super VL 120B-A12B locally?

You need a multi-GPU server. At Q4_K_M, Nemotron 3.5 Super VL 120B-A12B needs 81.9 GB VRAM, more than any single consumer GPU. That's roughly 2x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.

Can I run Nemotron 3.5 Super VL 120B-A12B in the cloud?

Yes. The cheapest RunPod setup we found that holds Nemotron 3.5 Super VL 120B-A12B entirely in GPU memory at 8k context is 1× RTX Pro 6000 at NVFP4, at roughly 83.2 tokens per second and from $1.69 per hour for the whole pod. Prices as of Oct 13, 2026; RunPod rates change, so check their pricing page for the current figure.

Can Nemotron 3.5 Super VL run on a single RTX Pro 6000 or an 80 GB H100?

A 96 GB RTX Pro 6000, yes: Q4_K_M totals 81.9 GB at 8k context and 84.3 GB at a 262,144-token window, and NVIDIA's NVFP4 checkpoint (84.1 GB at 8k) also fits, since the RTX Pro 6000 is a Blackwell card. The full 1,048,576-token window (91.5 GB at Q4_K_M) spills past what a 96 GB card holds. An 80 GB H100 or A100 cannot hold Q4_K_M, whose total is 81.9 GB at 8k, so one card is limited to Q3_K_M (64.7 GB); NVIDIA's own NVFP4 recipe uses two H100s or one 141 GB H200 instead.

Does Nemotron 3.5 Super VL run on NVIDIA's DGX Spark?

Yes, and it is one of the targets NVIDIA names for the NVFP4 checkpoint. This site's calculator puts that 75.0 GB checkpoint at 84.1 GB in total at 8k context, comfortably inside the Spark's 128 GB, at about 25.8 tokens per second given the Spark's 273 GB/s memory bandwidth. NVIDIA also publishes a separate LiLiCorr drafter for the Spark and says it generates text up to 2.3 times faster than decoding without speculation, a vendor figure this site has not reproduced.

Is there an Ollama or llama.cpp version of Nemotron 3.5 Super VL, and how big is it?

Both exist. Ollama lists the model as nemotron-3.5-super with text and image input and a 1M context; its default tag (ollama run nemotron-3.5-super:120b-a12b) is listed at 80 to 90 GB, its q4_K_M tag at 90 GB, and its q8_0 at 133 GB. For llama.cpp, ggml-org, Unsloth, and lmstudio-community publish GGUFs. Mind the sizes: the real two-shard Q4_K_M from lmstudio-community is 70.1 GB, Unsloth's recommended UD-Q4_K_XL is 86.9 GB, and Ollama's q4_K_M tag is about 20 GB bigger than the llama.cpp file. Vision needs a separate 1.7 GB mmproj file alongside the weights.

What can Nemotron 3.5 Super VL take as input?

Text, images, and video, with text output. NVIDIA's card lists RGB JPEG and PNG images and MP4 video of up to two minutes; its vLLM recipe samples video at 2 frames per second, up to 256 frames. There is no audio input: the model has no audio encoder. Images and video frames share the same 1,048,576-token window as the text, and reasoning tokens count against the max_tokens budget along with the answer, so leave room for both.

Nemotron 3.5 Super VL 120B-A12B

Name: Nemotron 3.5 Super VL 120B-A12B
Author: NVIDIA

Nemotron 3.5 Super VL 120B-A12B needs roughly 81.9 GB VRAM at Q4_K_M quantization (268.9 GB at FP16). 33 GPUs we track can run it fully in VRAM at 8k context.

33 GPUs run this natively · 11 with CPU offload

NVIDIA120B params12B active (MoE)1024k contextOpenMDW-1.1Commercial use ok

Q4_K_M total: 81.9 GB; at 8k context
Smallest GPU: 96 GB; NVIDIA RTX Pro 6000, at Q4_K_M
KV cache, full context: 8.6 GB; 8 of 88 layers cache
Inputs: Text · Image · Video; OpenMDW-1.1, released October 2026
Active experts: 22 of 512 + 1 shared; ~12B active per token
Caching layers: 8 of 88; Mamba-2 + MoE layers hold no growing KV cache

Nemotron 3.5 Super VL 120B-A12B is a Mixture of Experts (MoE) model with 120B total parameters but only 12B active per token developed by NVIDIA. Released 15 October 2026 under NVIDIA's OpenMDW-1.1 license as the vision-language successor to Nemotron 3 Super, initialized from Nemotron 3 Super Base and distilled from the larger Nemotron 3 Ultra. Every MoE layer holds 512 routed experts, 22 chosen per token plus 1 shared. The 88-layer stack interleaves 40 Mamba-2 layers, 40 MoE layers, and 8 attention layers (2 KV heads at head dimension 128), so only 8 layers keep a KV cache that grows with context. A C-RADIO vision encoder reads images, documents, and video of up to two minutes; output is text. The context window is 1,048,576 tokens, and reasoning can be switched off through the chat template.

To run Nemotron 3.5 Super VL 120B-A12B locally: Q4_K_M weights are 73.1GB by this site's estimate (the real llama.cpp file from lmstudio-community is 70.1GB), 81.9GB in total at 8k context: that fits a 96GB RTX Pro 6000 and any 128GB unified-memory machine (DGX Spark, M4 Max, M5 Max, Strix Halo), but not an 80GB H100 or A100, where Q3_K_M (64.7GB in total) is the largest build that fits. Only 8 of the 88 layers keep a KV cache, so long context is cheap: the full 1,048,576-token window adds 8.6GB, against 94.5GB if every layer cached. That puts Q4_K_M at 91.5GB at the full window, within reach of a 128GB machine but past a 96GB card's real ceiling, which holds it up to 262,144 tokens (84.3GB). NVIDIA's own NVFP4 checkpoint is 75.0GB (the vision encoder stays in BF16), 84.1GB in total at 8k; NVIDIA lists one DGX Spark, one H200, or two H100s for it, and four H100s or two GB200s for the 249GB BF16 checkpoint. As a MoE model, inference speed depends on active parameters (12B) rather than total size.

NVIDIA's own comparison table, collected through its NeMo Gym harness, has the model ahead of its Nemotron 3 Ultra teacher on GPQA Diamond (89.52 against 87.0), Humanity's Last Exam text-only (31.79 against 29.2), BrowseComp (78 against 67.85), and Terminal-Bench 2.1 (60.67 against 54.8), and behind it on SWE-bench Verified (71.10 against 73.9) and tau2-bench telecom (73.19 against 83.0). Against the similarly sized Qwen 3.5 122B-A10B it leads on browsing, tool use, and code (BrowseComp 78 against 58.1) but trails on charts and math diagrams (CharXiv reasoning 80.82 against 83.5, MathVision 78.16 against 83.3). Newer open models score higher on nearly every row: Qwen3.8-Flash-Next scores higher on 19 of the 20 agentic, coding, and vision rows in NVIDIA's own table, and Artificial Analysis, the independent number, puts the model at 26 on its Intelligence Index v4.3.2. The case for it is speed and fit rather than raw intelligence: NVIDIA reports up to 5 times the throughput of comparable open models on a single GPU, and a native multi-token-prediction head speeds up decoding without a separate drafter.

Only 8 of its 88 layers keep a growing memory

Nemotron 3.5 Super VL stacks three kinds of layer: 40 Mamba-2 layers, 40 mixture-of-experts layers (512 experts, 22 picked per token plus one shared expert), and 8 ordinary attention layers. Attention is the part that stores a key and a value for every token it has read, which is what makes a long conversation expensive. Mamba-2 keeps a fixed-size summary instead, and the expert layers never look back at all. In other words: eight layers keep a notebook that grows with the conversation, and the other eighty work from a memory that stays the same size. The strip below sorts the layers by whether they cache (grey for the 80 that do not, green for the 8 attention layers) rather than drawing them in their real order; in the real stack the kinds interleave, with an attention layer every nine to eleven layers (positions 8, 17, 26, 37, 48, 59, 70, and 79).

Mamba-2state-space, fixed-size recurrent stateMoE FFN512 routed experts, 22 active + 1 shared, no KVFull Attention2 KV heads × 128 dim, GQA

layer 140:40:8 pattern × 1layer 88

8 of 88 layers keep a KV cache that grows with the full context. The other 80 hold a fixed-size recurrent state, so their memory cost does not move as the context window grows.

A Mamba-2 layer updates its fixed-size state in place, so it costs the same whether the prompt is 2,000 tokens or 1,000,000. Only the 8 attention layers (2 KV heads of 128 dimensions each) accumulate a cache, at 8 KB per token. The vision encoder sits in front of this stack: images and video frames are turned into tokens that share the same context window as the text, so they count against the same cache.

What a full 1,048,576-token context actually costs

Long context is where most local setups give out: the cache grows with every token, and on a conventional 120B model it would outweigh the weights long before the window filled. Keeping the cache to 8 of 88 layers is what makes this model's 1M-token window affordable on a single 128 GB machine.

Nemotron 3.5 Super VL 120B-A12B: 8 of 88 layers cacheThe same 120B-A12B if all 88 layers used full attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Filling the full 1,048,576-token window costs 8.6 GB of KV cache instead of 94.5 GB. Add 73.1 GB of Q4_K_M weights and the full-context total is 91.5 GB: a 128 GB DGX Spark or Mac holds it, while a 96 GB RTX Pro 6000 holds Q4_K_M up to a 262,144-token window (84.3 GB) and starts to spill past that. The all-attention version of the same model would need 187.7 GB at the full window, more than any 128 GB machine or a 141 GB H200 can hold.

VRAM at each quantization

Nemotron 3.5 Super VL 120B-A12B natively supports a longer context window, but the table below is capped at 8k for comparability; its hybrid attention stack keeps KV cache growth well below linear.

Quant	Weights	KV cache	Total
FP32	480.0 GB	0.07 GB	537.7 GB
BF16	240.0 GB	0.07 GB	268.9 GB
FP16	240.0 GB	0.07 GB	268.9 GB
Q8_0	127.6 GB	0.07 GB	142.9 GB
Q6_K	98.5 GB	0.07 GB	110.4 GB
Q5_K_M	85.4 GB	0.07 GB	95.8 GB
Q4_K_Mrec	73.1 GB	0.07 GB	81.9 GB
Q3_K_M	57.7 GB	0.07 GB	64.7 GB
Q2_K	45.7 GB	0.07 GB	51.3 GB
NVFP4cuda	75.0 GB	0.07 GB	84.1 GB

KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.

Quantization × context: where it fits

The table above holds context fixed at 8k. Add context as a second axis and the hybrid stack shows: even the widest jump, from 8k to the full window, adds under 10 GB to any row, so the quantization you pick decides what fits far more than how much context you use.

Total VRAM needed by quantization and context length, with the GPU class each figure requires.
Quant	8k ctx	32k ctx	128k ctx	1024k ctx
Q8_0	142.9 GB80 GB+ card	143.2 GB80 GB+ card	144.1 GB80 GB+ card	152.5 GB80 GB+ card
Q6_K	110.4 GB80 GB+ card	110.6 GB80 GB+ card	111.5 GB80 GB+ card	120.0 GB80 GB+ card
Q5_K_M	95.8 GB80 GB+ card	96.0 GB80 GB+ card	96.9 GB80 GB+ card	105.3 GB80 GB+ card
Q4_K_Mrec	81.9 GB80 GB+ card	82.2 GB80 GB+ card	83.0 GB80 GB+ card	91.5 GB80 GB+ card
Q3_K_M	64.7 GB80 GB card	65.0 GB80 GB card	65.8 GB80 GB card	74.3 GB80 GB card

Each cell shows the smallest card that holds it:Datacenter GPU

Benchmarks

89.5

88.2

71.1

60.7

On GPQA Diamond, a 120B model above its 550B teacher

On NVIDIA's own GPQA Diamond number (89.52), this 12B-active model sits above its 550B teacher, Nemotron 3 Ultra (87.0), and above the similarly sized Qwen 3.5 122B-A10B (86.6). It is not at the top: 11 other models on this site score higher, every one of them larger, led by Kimi K3's 93.5. In other words: distillation moved a lot of the teacher's science-question ability into a model about a fifth of the size, but it did not close the gap to the very largest releases.

GPQA Diamond, as reported by each model's own publisher. Scaffolding and harness differ between vendors, so read the gaps as approximate rather than a ranking.

Against the nearest-sized Qwen: stronger at agent work, weaker at reading charts

Qwen 3.5 122B-A10B (122B total, 10B active) is the closest model by size in NVIDIA's own comparison table and on this site's board, so it is the fairest reference. Across all 36 rows the table lists for both, the new model is ahead on 22. Narrow that to the agentic, coding, and vision rows and the split is sharp: ahead on 8 of 11 agentic and coding rows, but on only 2 of 9 vision rows, the weaker side for a vision-language model. The chart shows the five biggest leads and the five biggest deficits among those 20 rows.

BrowseComp

Super VL

78.0

Qwen 3.5 122B

58.1

tau2-bench retail

Super VL

69.2

Qwen 3.5 122B

55.4

tau2-bench airline

Super VL

76.5

Qwen 3.5 122B

63.5

Terminal-Bench 2.1

Super VL

60.7

Qwen 3.5 122B

48.7

LiveCodeBench v6

Super VL

88.2

Qwen 3.5 122B

76.7

tau2-bench telecom

Super VL

73.2

Qwen 3.5 122B

87.2

OCRBench v2 (ZH)

Super VL

59.1

Qwen 3.5 122B

67.3

BabyVision

Super VL

29.6

Qwen 3.5 122B

35.7

Super VL

85.6

Qwen 3.5 122B

91.1

MathVision

Super VL

78.2

Qwen 3.5 122B

83.3

NVIDIA's own comparison table on the BF16 model card, collected through its NeMo Gym harness; the vision rows (BabyVision, V*, MathVision, OCRBench v2, and the other image and video sets) used official open-source implementations or NVIDIA's internal scaffolding instead. The 20 rows are every agentic, coding, and vision score on a 0 to 100 scale where both models are listed; GDPval-AA (an Elo-style score) and the Artificial Analysis index rows are left out. Qwen3.8-Flash-Next, also in the table, scores higher than Nemotron on 19 of the same 20 rows, so this is a fair match for size, not for strength. Nothing here is independently reproduced; Artificial Analysis's independent Intelligence Index v4.3.2 scores the model 26.

The leads are all in agent work: browsing, multi-turn tool use in retail and airline settings, terminal tasks, and code, by 12 to 20 points. The deficits cluster in vision and in one tool-use set, tau2-bench telecom, where it trails by 14 points. If your workload is mostly documents and charts, test a Qwen model on your own pages before choosing; if it is browsing and tool calling, this is the stronger of the two.

GPUs that run Nemotron 3.5 Super VL 120B-A12B natively (33)

NVIDIA B300 288GBBF16 · 50.3 t/s
NVIDIA B200 180GBNVFP4 · 89.3 t/s
NVIDIA H200 141GBQ6_K · 74.7 t/s
NVIDIA H100 80GBQ3_K_M · 87.4 t/s
NVIDIA A100 80GBQ3_K_M · 78.6 t/s

Show 28 more

NVIDIA RTX Pro 6000NVFP4 · 83.2 t/s
NVIDIA DGX Spark (128GB)NVFP4 · 25.8 t/s
AMD Instinct MI300XQ8_0 · 42.5 t/s
AMD Strix Halo (128GB)Q6_K · 16.7 t/s
AMD Strix Halo (96GB)Q4_K_M · 20.6 t/s
AMD Strix Halo (64GB)Q2_K · 27.4 t/s
Apple M5 Ultra (512GB)BF16 · 22.5 t/s
Apple M5 Ultra (256GB)Q8_0 · 30.6 t/s
Apple M5 Ultra (96GB)Q4_K_M · 37 t/s
Apple M5 Max (128GB)Q6_K · 25.3 t/s
Apple M5 Max (64GB)Q2_K · 34.7 t/s
Apple M5 Pro (64GB)Q2_K · 26.1 t/s
Apple M4 Max (128GB)Q6_K · 23.8 t/s
Apple M4 Max (64GB)Q2_K · 33.4 t/s
Apple M3 Ultra (512GB)BF16 · 17.8 t/s
Apple M3 Ultra (256GB)Q8_0 · 25.7 t/s
Apple M3 Ultra (96GB)Q4_K_M · 32.7 t/s
Apple M3 Max (128GB)Q6_K · 19.9 t/s
Apple M3 Max (96GB)Q4_K_M · 20 t/s
Apple M3 Max (64GB)Q2_K · 29.5 t/s
Apple M2 Ultra (192GB)Q8_0 · 25.4 t/s
Apple M2 Ultra (64GB)Q2_K · 37.6 t/s
Apple M2 Max (96GB)Q4_K_M · 23.6 t/s
Apple M2 Max (64GB)Q2_K · 29.5 t/s
Apple M1 Ultra (128GB)Q6_K · 28.7 t/s
Apple M1 Ultra (64GB)Q2_K · 37.6 t/s
Apple M1 Max (64GB)Q2_K · 29.5 t/s
Intel Data Center GPU Max 1550Q6_K · 43.3 t/s

Plus 11 GPUs that run it with CPU offload (slower)

NVIDIA RTX 5090Q2_K · 20.5 t/s
NVIDIA A100 40GBQ2_K · 27.3 t/s
NVIDIA L40SQ3_K_M · 19.4 t/s
NVIDIA RTX A6000Q3_K_M · 19.8 t/s
NVIDIA RTX 5000 AdaQ2_K · 18.6 t/s
NVIDIA RTX 6000 AdaQ3_K_M · 20.4 t/s
AMD Radeon PRO W7800Q2_K · 16 t/s
AMD Radeon PRO W7900Q3_K_M · 17.2 t/s
AMD Radeon AI PRO R9700 32GBQ2_K · 16.2 t/s
Intel Arc Pro B70 32GBQ2_K · 16.1 t/s
Intel Data Center GPU Max 1100Q3_K_M · 17.2 t/s

Cloud GPU Rental

Cheapest way to run Nemotron 3.5 Super VL 120B-A12B: 1× RTX Pro 6000 at NVFP4, ~83 tok/s, from $1.69/hr.

Pay by the hour · no contract · pods start in about a minute.

Rent this setup on RunPod ↗ (+$5 signup credit)

Prices as of Oct 13, 2026. Rates change; check RunPod for current pricing.

Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

NVIDIA's open vision-language agent model, released on 15 October 2026 under OpenMDW-1.1 as the multimodal successor to Nemotron 3 Super. Hybrid Mamba-2 / MoE / attention stack with 512 routed experts (22 active per token plus 1 shared) and a 1M-token context window; only 8 of its 88 layers keep a KV cache that grows with context. Initialized from Nemotron 3 Super Base, distilled from Nemotron 3 Ultra, and given a C-RADIO vision encoder for images, documents, and video of up to two minutes; output is text. Reasoning can be switched off with enable_thinking=False, and a native multi-token-prediction head speeds up decoding. Benchmark figures are NVIDIA's own, from the BF16 model card: it beats Nemotron 3 Ultra on GPQA Diamond (89.52 against 87.0), Humanity's Last Exam text-only (31.79 against 29.2), BrowseComp (78 against 67.85), and Terminal-Bench 2.1 (60.67 against 54.8), but trails it on SWE-bench Verified (71.10 against 73.9) and tau2-bench telecom (73.19 against 83.0), and trails Qwen3.8-Flash-Next, DeepSeek V4 Flash, and GLM-5.3-Flash on most agentic rows. Artificial Analysis scores it 26 on its Intelligence Index v4.3.2, so the pitch is throughput and document understanding rather than raw intelligence. Real download sizes: the BF16 checkpoint is 249 GB, NVIDIA's NVFP4 checkpoint is 75.0 GB, and community GGUFs run from 46 GB (Unsloth UD-IQ1_M) through 70.1 GB (Q4_K_M) to 128.5 GB (Q8_0).

Hugging Face ↗Ollama ↗Released 2026-10-15

How to run Nemotron 3.5 Super VL 120B-A12B locally

816244880160320

Q4_K_M needs 81.9 GB: needs multiple datacenter-class GPUs (80 GB+ each).

vLLM with NVIDIA's NVFP4 checkpoint (two H100s or one H200; DGX Spark has its own recipe)

vllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
  --served-model-name nemotron-3.5-super-vl \
  --tensor-parallel-size 2 \
  --max-num-seqs 128 \
  --trust-remote-code \
  --gpu-memory-utilization 0.89 \
  --max-num-batched-tokens 16384 \
  --block-size 64 \
  --prefix-match-unit 64 \
  --compilation_config.max_cudagraph_capture_size 256 \
  --mamba-backend flashinfer \
  --mamba-ssm-cache-dtype float16 \
  --enable-mamba-cache-stochastic-rounding \
  --mamba-cache-philox-rounds 5 \
  --enable-mamba-shared-prefix-checkpoint \
  --mamba-cache-mode align \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 3 \
  --video-pruning-rate 0.5 \
  --media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}' \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3
# From NVIDIA's NVFP4 model card. One H200: the same command with --tensor-parallel-size 1.
# Use the vllm/vllm-openai:v0.32.0 image (or the nightly the card pins).
# DGX Spark uses a different recipe with the LiLiCorr drafter; see the card.

Ollama

ollama run nemotron-3.5-super:120b-a12b

llama.cpp

# Q4_K_M in two shards (70.1 GB). Vision needs the mmproj file from the same repo;
# if your build does not fetch it for you, add --mmproj mmproj-NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16.gguf
llama-server \
  -hf lmstudio-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF:Q4_K_M \
  --ctx-size 16384 \
  --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0.0

# Unsloth recommends its UD-Q4_K_XL (86.9 GB) for the best accuracy per gigabyte; same flags.
# Model card sampling: temperature 1.0 and top_p 0.95 for text, temperature 0.6 and top_p 0.95
# for vision. Reasoning is on by default; enable_thinking=False in the chat template turns it off.
# Start at 16K context and raise it: the 1M window is a ceiling, not a default.

LM Studio: Search for 'Nemotron 3.5 Super VL' and take lmstudio-community's Q4_K_M (two shards, 70.1 GB); the mmproj file in the same repo enables image input. NVIDIA's own cards document vLLM, SGLang, TensorRT-LLM, NIM, and Dynamo, and point local users on DGX Spark and RTX Pro 6000 to ggml-org's llama.cpp builds and Unsloth. Whether LM Studio's own catalog lists the model, and whether it handles video, is unconfirmed here.

Why this quantization? Q4_K_M puts the weights at 73.1 GB by this site's estimate (the real two-shard llama.cpp file is 70.1 GB; Unsloth's UD-Q4_K_XL is 86.9 GB, and Ollama lists its q4_K_M tag at 90 GB), and the total is 81.9 GB at 8k context. Only 8 of the 88 layers keep a KV cache, so the total barely moves as context grows: 84.3 GB at 262,144 tokens and 91.5 GB with the full 1,048,576-token window filled. That holds up to a 262,144-token window on a 96 GB RTX Pro 6000 and the whole window on a 128 GB DGX Spark or Mac. An 80 GB H100 or A100 cannot hold Q4_K_M (81.9 GB at 8k), and Q3_K_M (64.7 GB) is the largest build that fits there. On Blackwell hardware NVIDIA's own NVFP4 checkpoint is the better choice: 75.0 GB of weights and 84.1 GB in total at 8k, trained with quantization-aware distillation, and NVIDIA's table shows it within about a point of BF16 on most rows. The exceptions are the agentic rows WebVoyager (77.72 to 66.31) and DeepSWE (24.65 to 18.2), and MMLU-ProX (76.91 to 73.76), so check those if your workload leans on them.

Who is Nemotron 3.5 Super VL 120B-A12B for?

Owners of a 128 GB unified-memory machine (DGX Spark, an M4 or M5 Max Mac, Strix Halo), a 96 GB RTX Pro 6000, or a pair of H100s who want an open model that reads images, documents, and short video and also works as a tool-calling agent, and who can accept that newer open models score higher on most benchmarks.

Best for

Document intelligence: charts, tables, scanned pages, and multi-page PDFs rendered to images (NVIDIA reports CharXiv reasoning 80.82 and OCRBench v2 English 68.12)
Browsing and terminal agents: NVIDIA's table has BrowseComp 78 and Terminal-Bench 2.1 60.67, ahead of the similarly sized Qwen 3.5 122B-A10B
Fine-tuning on private data: NVIDIA also releases the base checkpoint, training recipes, and RL environments alongside the post-trained weights
Long-context work on one box: the full 1,048,576-token window adds only 8.6 GB of KV cache
Summarizing or searching short video clips locally, up to two minutes per clip

Not ideal for

Anyone who wants the strongest open model: Qwen3.8-Flash-Next leads NVIDIA's own table on nearly every row, and Artificial Analysis scores this model 26 on its Intelligence Index v4.3.2
80 GB cards (H100, A100) at Q4_K_M: 81.9 GB at 8k context, so only Q3_K_M (64.7 GB) fits one card; NVIDIA's own two-card recipe is the alternative
96 GB cards running Ollama's own q4_K_M tag: listed at 90 GB, it leaves no room for context; use the 70.1 GB llama.cpp file instead
Audio input: the model has no audio tower, only text, image, and video
Latency-sensitive chat: reasoning is on by default, and every reasoning token counts against the same max_tokens budget as the answer

So should you run it?

Yes, if you own a 128 GB machine or a 96 GB workstation card and want an open model that reads images, documents, and short video as well as it calls tools. Q4_K_M's 73.1 GB of weights total 81.9 GB at 8k context, so a 96 GB RTX Pro 6000 runs it at 84.1 tokens per second by this site's estimate with room for a 262,144-token window, and a DGX Spark, M5 Max, or M4 Max holds even the full million tokens (91.5 GB), at 13.6, 19.3, and 17.9 tokens per second at that length. An 80 GB H100 or A100 cannot hold Q4_K_M and is limited to Q3_K_M (64.7 GB in total). NVIDIA's own recipes use its NVFP4 checkpoint (75.0 GB, 84.1 GB in total at 8k) on Hopper as well as Blackwell cards, though this site's calculator only offers NVFP4 on Blackwell. On quality, treat it as a capable specialist rather than the best general model: NVIDIA's own table has Qwen3.8-Flash-Next ahead on 19 of 20 agentic, coding, and vision rows, and Artificial Analysis's independent index puts it at 26. The reasons to pick it anyway are the open recipe and data, the license NVIDIA calls ready for commercial use, the cheap long context, and speed: NVIDIA reports up to 5 times the throughput of comparable open models on one GPU, a vendor figure this site's calculator cannot check.

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Nemotron 3.5 Super VL 120B-A12B?: Nemotron 3.5 Super VL 120B-A12B requires approximately 81.9 GB of VRAM at Q4_K_M quantization, 142.9 GB at Q8, and 268.9 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Nemotron 3.5 Super VL 120B-A12B have?: Nemotron 3.5 Super VL 120B-A12B has 120 billion total parameters, but only 12 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
Is Nemotron 3.5 Super VL 120B-A12B good for coding?: Yes. Nemotron 3.5 Super VL 120B-A12B scores 88.2 on LiveCodeBench, demonstrating strong code generation and completion capabilities.
Can Nemotron 3.5 Super VL 120B-A12B run on a 16 GB GPU?: No. At Q4_K_M, Nemotron 3.5 Super VL 120B-A12B needs 81.9 GB of VRAM, more than 16 GB. You will need a multi-GPU server.
Can Nemotron 3.5 Super VL 120B-A12B run on a 24 GB GPU?: No. Even at Q4_K_M, Nemotron 3.5 Super VL 120B-A12B needs 81.9 GB. Consider a multi-GPU server with 82 GB+ of combined VRAM.
What is the smallest quantization for Nemotron 3.5 Super VL 120B-A12B that fits in 24 GB of VRAM?: Nemotron 3.5 Super VL 120B-A12B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 51.3 GB at Q2_K.
What GPU do I need to run Nemotron 3.5 Super VL 120B-A12B locally?: You need a multi-GPU server. At Q4_K_M, Nemotron 3.5 Super VL 120B-A12B needs 81.9 GB VRAM, more than any single consumer GPU. That's roughly 2x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.
Can I run Nemotron 3.5 Super VL 120B-A12B in the cloud?: Yes. The cheapest RunPod setup we found that holds Nemotron 3.5 Super VL 120B-A12B entirely in GPU memory at 8k context is 1× RTX Pro 6000 at NVFP4, at roughly 83.2 tokens per second and from $1.69 per hour for the whole pod. Prices as of Oct 13, 2026; RunPod rates change, so check their pricing page for the current figure.
Can Nemotron 3.5 Super VL run on a single RTX Pro 6000 or an 80 GB H100?: A 96 GB RTX Pro 6000, yes: Q4_K_M totals 81.9 GB at 8k context and 84.3 GB at a 262,144-token window, and NVIDIA's NVFP4 checkpoint (84.1 GB at 8k) also fits, since the RTX Pro 6000 is a Blackwell card. The full 1,048,576-token window (91.5 GB at Q4_K_M) spills past what a 96 GB card holds. An 80 GB H100 or A100 cannot hold Q4_K_M, whose total is 81.9 GB at 8k, so one card is limited to Q3_K_M (64.7 GB); NVIDIA's own NVFP4 recipe uses two H100s or one 141 GB H200 instead.
Does Nemotron 3.5 Super VL run on NVIDIA's DGX Spark?: Yes, and it is one of the targets NVIDIA names for the NVFP4 checkpoint. This site's calculator puts that 75.0 GB checkpoint at 84.1 GB in total at 8k context, comfortably inside the Spark's 128 GB, at about 25.8 tokens per second given the Spark's 273 GB/s memory bandwidth. NVIDIA also publishes a separate LiLiCorr drafter for the Spark and says it generates text up to 2.3 times faster than decoding without speculation, a vendor figure this site has not reproduced.
Is there an Ollama or llama.cpp version of Nemotron 3.5 Super VL, and how big is it?: Both exist. Ollama lists the model as nemotron-3.5-super with text and image input and a 1M context; its default tag (ollama run nemotron-3.5-super:120b-a12b) is listed at 80 to 90 GB, its q4_K_M tag at 90 GB, and its q8_0 at 133 GB. For llama.cpp, ggml-org, Unsloth, and lmstudio-community publish GGUFs. Mind the sizes: the real two-shard Q4_K_M from lmstudio-community is 70.1 GB, Unsloth's recommended UD-Q4_K_XL is 86.9 GB, and Ollama's q4_K_M tag is about 20 GB bigger than the llama.cpp file. Vision needs a separate 1.7 GB mmproj file alongside the weights.
What can Nemotron 3.5 Super VL take as input?: Text, images, and video, with text output. NVIDIA's card lists RGB JPEG and PNG images and MP4 video of up to two minutes; its vLLM recipe samples video at 2 frames per second, up to 256 frames. There is no audio input: the model has no audio encoder. Images and video frames share the same 1,048,576-token window as the text, and reasoning tokens count against the max_tokens budget along with the answer, so leave room for both.