NVIDIA RTX 4090 vs NVIDIA RTX 3090

Analysis

The RTX 4090 vs RTX 3090 is one of the most searched local-LLM GPU comparisons on this site, and for good reason: both cards top out at the exact same 24GB of VRAM, so the real question isn't which one runs bigger models, it's how much extra speed two years of Ada Lovelace bought over Ampere, and whether the RTX 3090's one lasting exclusive, a working NVLink bridge, is worth chasing on the used market.

Because both cards share an identical 24GB ceiling, this site's calculator fits the exact same 52 of the 94 models this site tracks natively on either card at 8k context: VRAM capacity, not bandwidth, decides what fits, and here the two are tied. The entire difference is speed: the RTX 4090's 1,008 GB/s is 7.7% faster than the RTX 3090's 936 GB/s, and this site's calculator shows that gap landing almost exactly in tokens per second on every model both cards fit natively: Llama 3.1 8B at Q4_K_M decodes at 110.2 tok/s on the RTX 4090 versus 102.3 tok/s on the RTX 3090, GPT-OSS 20B at Q4_K_M runs 87.2 tok/s versus 81.0 tok/s, and Qwen 3.6 27B at its recommended Q5_K_M runs 33.2 tok/s versus 30.8 tok/s, each pair landing close to that same 7.7% ratio. Qwen3 32B and Yi 1.5 34B both need CPU offload at Q4_K_M on either card (23.88 GB and 25.72 GB against roughly 22.8 GB of usable VRAM), and the ratio holds even under offload: 30.7 tok/s on the RTX 4090 versus 28.5 tok/s on the RTX 3090 for Qwen3 32B. The RTX 4090's 16,384 CUDA cores, cut down from an AD102 die, add 4th-gen Tensor Cores with hardware FP8 support that Ampere's 3rd-gen Tensor Cores lack, though using it takes TensorRT-LLM or vLLM rather than the GGUF quantization most local setups run, so it doesn't show up in the numbers above. What the RTX 3090 keeps that the RTX 4090 lost: NVIDIA removed the NVLink connector starting with Ada Lovelace, so the RTX 3090 (and RTX 3090 Ti) are the last GeForce cards that can bridge two GPUs into one address space. In practice that buys less than it sounds like for llama.cpp's default layer-split inference, which barely needs the bridge at all: two RTX 4090s pooled over plain PCIe still decode Llama 3.3 70B at Q3_K_M faster than two NVLink-bridged RTX 3090s (16.2 tok/s versus 15.1 tok/s), purely from the RTX 4090's own higher bandwidth, no bridge required.

Bottom line: For anyone buying new or choosing between similarly-priced used cards, the RTX 4090 is the better single-card pick: identical VRAM capacity, a real 7.7% speed edge on everything both cards fit, and hardware FP8 support Ampere never got. The RTX 3090 earns its keep specifically on price: two RTX 3090s are usually cheaper on the used market than one RTX 4090, and a used 3090 pair is still the most accessible way to reach 48GB of pooled VRAM for 70B-class models. Don't buy the NVLink bridge expecting a decode-speed bonus over plain multi-GPU layer-splitting; the honest reason to pick RTX 3090s over RTX 4090s is capacity per dollar on the used market, not NVLink itself.