NVIDIA RTX 5090

NVIDIA RTX 5090: This is the highest-bandwidth desktop GPU this site tracks (1,792 GB/s, 77.8% more than the RTX 4090's 1,008 GB/s) from GDDR7 replacing GDDR6X on a similar-width bus rather than a wider one (Wikipedia's GeForce RTX 50 series spec table, cross-checked against NVIDIA's own product page). The bigger story eighteen months after launch is the price: an AI-datacenter-driven GDDR7 supply crunch pushed real street prices to $4,300-$4,600 by mid-2026 (videocardprices.com tracking, August 2026), more than double the $1,999 launch MSRP, a genuinely strange irony for an AI-capable consumer card, since the same AI boom that makes it desirable for local inference is also what's diverting the memory supply that would otherwise go into building more of them.

This site's calculator puts Qwen3 32B at its recommended NVFP4 quant at 19.87 GB (65.7 tok/s), the largest dense model that fits this card natively with real headroom at 8k context, and Llama 3.1 8B at Q4_K_M at 195.9 tok/s, 77.8% faster than the same model's 110.2 tok/s on an RTX 4090, tracking the bandwidth gain almost exactly. 70B-class dense models (Llama 3.3 70B, Qwen 2.5 72B, Llama 3.1 70B) don't fit natively at any quantization at 8k context: even Llama 3.3 70B's smallest real build (NVFP4, 42.21 GB) and its more common Q4_K_M build (50.75 GB) both exceed this card's roughly 30 GB of usable VRAM, so it lands in the offload bucket at 1.6-3.1 tok/s rather than running natively. GPT-OSS 120B doesn't fit at all, even with full system-RAM offload: its 62.6 GB MXFP4 weight floor alone is close to this card's entire usable VRAM+RAM budget. Community benchmarks (hardware-corner.net) report comparable native-fit throughput in practice: Qwen3 8B generating at 185.91 tok/s and prefill above 10,000 tok/s at 4K context, though real llama.cpp batching means these aren't directly the same measurement as this site's steady-state estimate.

Full CUDA support, but Blackwell's sm_120 compute capability was new enough at launch that official PyTorch wheels lagged for a long stretch: a GitHub issue asking PyTorch to add stable sm_120 support (pytorch/pytorch#159207) was still open in July 2025, six months after this card shipped, forcing early PyTorch-based tooling to build from source or use nightlies. llama.cpp-based tools (Ollama, LM Studio) mostly avoided that gap since llama.cpp compiles its own CUDA kernels. NVIDIA's own engineering blog reports up to 35% higher throughput from CUDA graph enablement and up to 15% from flash-attention kernels after upgrading to the CUDA 12.8 runtime Blackwell requires (a specific 27% LM Studio speedup was measured on the related RTX 5080, not this card), the software stack has matured substantially since, but a very old llama.cpp, Ollama, or PyTorch build still won't recognize this GPU's architecture. Native NVFP4 support is the other software-visible difference from Ada/Ampere: it's the only quantization format in this site's ladder that requires Blackwell specifically.