NVIDIA RTX 50 Series (Blackwell) for LLMs: Complete Guide

Blackwell Architecture: What Changed for LLMs

NVIDIA's RTX 50 series (Blackwell, 2025-2026) brings three meaningful improvements for LLM inference. First, GDDR7 memory delivers significantly higher bandwidth: the RTX 5090's 1792 GB/s is a 78% increase over the 4090's 1008 GB/s. This directly translates to 1.5-2x faster token generation for same-size models. Second, the consumer VRAM ceiling rises to 32 GB (RTX 5090), up from 24 GB (RTX 4090). While still short of the 40 GB needed for 70B at Q4, 32 GB enables 70B at Q3_K_M (~31 GB) — the first time a single consumer card can technically run a 70B model. Third, NVFP4 (native 4-bit floating point) support via 5th-gen Tensor Cores. Unlike GGUF's integer quantization, NVFP4 uses a true 4-bit floating point representation that can preserve more model quality at the same bit-width. The caveat: NVFP4 requires software support that is still maturing. llama.cpp's NVFP4 backend is under development, and Ollama does not yet support it. For now, GGUF Q4_K_M remains the practical standard even on Blackwell. As the software ecosystem catches up, NVFP4 could become a meaningful differentiator. For production inference today, treat NVFP4 as a future-looking feature, not a current reason to buy.

RTX 5090 (32 GB): The New Flagship

The RTX 5090 (32 GB, 1792 GB/s, $2000 MSRP) is the most capable consumer LLM GPU ever made. Thirty-two gigabytes changes the game: for the first time, a consumer card can load a 70B model (Llama 3.3 70B at Q3_K_M, ~31 GB) with short context. At Q4 (~40 GB), 70B still does not fit — you need 48 GB. But Q3_K_M 70B on a single card is a milestone. For 27-32B models at Q4 (~17-20 GB), the 5090 delivers 55-70+ tok/s — roughly 1.8x faster than the 4090. This speed is transformative for agentic coding workflows where sessions can generate 20K+ tokens: a task that takes 10 minutes on the 4090 takes 5-6 minutes on the 5090. The 512-bit memory bus with 28 Gbps GDDR7 modules achieves the 1792 GB/s figure. NVFP4 tensor core support may eventually improve quality for 4-bit models, but GGUF remains the practical standard. For users building the best possible single-card local LLM rig, the RTX 5090 is the answer. The main argument against it: at $2000, you could instead buy dual RTX 3090s (48 GB, ~$1500-1800) and run 70B at Q4 — a configuration that needs more power and complexity but delivers better large-model capability.

# RTX 5090: first consumer card to run 70B
o llama run llama3.3:70b-q3_K_M    # ~31 GB, fits with short context

# 32B models at unprecedented speed
ollama run qwen3.6:27b              # ~17 GB, 55-70 tok/s
ollama run deepseek-r1:32b-q4_K_M   # ~20 GB, 45-55 tok/s

# NVFP4 support (when software matures)
# Will offer better quality than Q4_K_M at similar VRAM

RTX 5080 (24 GB), 5070 Ti (16 GB), 5070 (12 GB)

The RTX 5080 (24 GB, 1024 GB/s, $1000-1200 expected) matches the RTX 4090's VRAM at roughly 60% of the price. For LLM inference, it is essentially a 4090-class card at a lower price — 24 GB runs 32B models at Q4, with bandwidth slightly above the 4090. If pricing holds at $1000-1200, this becomes the new mid-range value champion for LLMs, potentially dethroning the used RTX 3090 for those who want new-with-warranty. The RTX 5070 Ti (16 GB, ~896 GB/s expected, $750-800) continues the pattern of 70-class cards at 16 GB. Excellent for 14B models, good bandwidth, but 16 GB limits model capacity. The RTX 5070 (12 GB, 672 GB/s, $549) is a fast 12 GB card: great bandwidth in its class, but 12 GB caps you at 14B models. For LLMs, 12 GB at $549 is poor value compared to the used RTX 3060 12GB ($200) or Arc B580 12GB ($249). The 5070 is a gaming card. The RTX 5060 Ti 16GB (~$500 expected, specs TBD) could be interesting if it pairs 16 GB with GDDR7 bandwidth significantly above the 4060 Ti's 288 GB/s. Early specifications suggest a 128-bit bus, which would cap bandwidth frustratingly low again.

NVFP4: The Blackwell Wildcard

NVFP4 is NVIDIA's hardware-accelerated 4-bit floating-point format, exclusive to Blackwell GPUs. Unlike GGUF's integer quantization (which maps continuous weight values to discrete integer buckets), NVFP4 uses a true miniaturized floating-point representation: 1 sign bit, 2 exponent bits, 1 mantissa bit. This floating-point approach can theoretically preserve more nuanced weight information than integer quantization at the same bit-width. The Blackwell tensor cores execute NVFP4 operations natively, meaning inference using NVFP4-formatted models can be both memory-efficient (4 bits per weight, same as Q4_K_M) and fast (tensor cores handle the computation rather than CUDA cores). The catch in mid-2026: software support is nascent. llama.cpp's NVFP4 backend is experimental and not yet merged into the main branch. Ollama does not support NVFP4 models. ExLlamaV2 has preliminary NVFP4 support for some architectures. NVFP4-formatted models on Hugging Face are rare — the ecosystem has not yet produced a broad library of pre-quantized NVFP4 models. For early adopters, NVFP4 is a reason to be excited about Blackwell's future. For production inference today, it is not a factor in purchase decisions. By late 2026 or early 2027, NVFP4 could become a standard option alongside GGUF Q4_K_M.

Which Blackwell GPU to Buy for LLMs

RTX 5090 ($2000): buy if you want the single best consumer GPU for LLMs and have the budget. The 32 GB + 1792 GB/s combination is unmatched. You will run 27-32B models at incredible speed and can touch 70B at Q3. RTX 5080 ($1000-1200): buy if you want 24 GB with warranty at a fair price. It matches the 4090's VRAM and slightly exceeds its bandwidth. This is the sensible enthusiast choice. RTX 5070 Ti 16GB ($750-800): buy if you need a new 16 GB NVIDIA card for 14B models with strong bandwidth. It is the best 14B-class card in the Blackwell lineup. RTX 5070 12GB ($549): skip for LLMs. Used alternatives (3090, 3060 12GB) are better. RTX 5060 Ti 16GB (~$500): wait for independent bandwidth benchmarks. If bandwidth is still 128-bit (~288 GB/s), skip — the RTX 3060 12GB is faster. If NVIDIA gives it a wider bus, it could be compelling. RTX 5050 and below: skip. Insufficient VRAM for practical LLM inference.

Blackwell (50 Series) vs Ada (40 Series): Upgrade Worth It?

From RTX 4090 to 5090: the upgrade is significant. 24→32 GB opens up 70B at Q3, and 1008→1792 GB/s roughly doubles generation speed. It is worth the upgrade if you run large models daily. From RTX 4080/4070 Ti Super to 5080: 16→24 GB is a meaningful VRAM upgrade that unlocks the 32B model class. The bandwidth improvement (717→1024 or 672→1024 GB/s) is also substantial. Worth it if you want access to 32B models. From RTX 4070/4060 Ti to 5070/5060 Ti: the generation-to-generation VRAM capacities are unchanged (12→12, 16→16). The bandwidth improvements from GDDR7 help speed but do not unlock new model sizes. Not worth upgrading for LLMs alone. From RTX 3090/3080 (Ampere) to 5080/5070 Ti: if you have a 3090 (24 GB), the 5080 (24 GB) is a side-grade in VRAM with a bandwidth bump (936→1024). Not transformative. The 5070 Ti (16 GB) is a VRAM downgrade from the 3090 (24 GB). From anything older (20 series, 10 series, GTX): any Blackwell above the 5060 is a massive upgrade. The generational leaps in VRAM, bandwidth, and CUDA features are transformative.