NVIDIA RTX 3090

NVIDIA RTX 3090: Launched September 24, 2020 at a $1,499 MSRP, the RTX 3090 was the flagship of NVIDIA's Ampere generation and the first GeForce card to carry 24GB of VRAM (NVIDIA's own RTX 3090/3090 Ti product page confirms every core spec here, cross-checked against Wikipedia's GeForce RTX 30 series spec table). Its GA102 die is cut down to 10,496 of the silicon's 10,752 CUDA cores (82 of 84 streaming multiprocessors), one bin below the fully unlocked GA102 configuration NVIDIA reserved for the RTX 3090 Ti and the professional RTX A6000. Five years after launch it still commands real money on the used market: community-tracked listings in 2026 commonly cite roughly $700 to $1,050 for a working card, evidence of sustained demand from local-AI hobbyists rather than the price a five-year-old flagship GPU would normally fetch on its own. The RTX 3090 and RTX 3090 Ti are also the last GeForce cards NVIDIA shipped with a working NVLink bridge: NVIDIA's own spec page lists NVLink support for both, and a bridge (commonly cited around 112.5 GB/s of bidirectional bandwidth, though NVIDIA's own product page doesn't publish that figure directly) lets two cards pool memory into one address space. The RTX 4090 and RTX 5090 that followed dropped the connector entirely, which is the real reason a used RTX 3090 pair, not a newer single card, is still the cheapest way into a 48GB pool for local inference.

This site's calculator puts Qwen3 8B at 100.1 tok/s at Q4_K_M (6.81 GB, fits with room to spare) and Qwen3 14B at its recommended Q8_0 quant at 35.6 tok/s (19.12 GB), both comfortably inside the 24GB budget with headroom for context. Qwen 3.8 27B fits too: its recommended Q4_K_M build leaves real headroom (19.02 GB, 35.8 tok/s), and even the higher-quality Q5_K_M still fits, just with well under a gigabyte of this card's ~22.8 GB usable budget left over (22.13 GB, 30.8 tok/s). The real ceiling sits around 32-34B, the same pattern the RTX 4090 shows one generation later: Qwen3 32B and Yi 1.5 34B both need Q3_K_M to fit natively (19.17 GB at 35.5 tok/s, and 20.79 GB at 32.8 tok/s) at 8k context, while the more common Q4_K_M build tips both into CPU offload (23.88 GB at 28.5 tok/s, and 25.72 GB at 22.7 tok/s), a modest falloff rather than a cliff. MoE models do better: Qwen3 30B-A3B fits at Q4_K_M (21.36 GB, 88.2 tok/s) and Qwen3.5 35B-A3B fits at Q3_K_M (19.04 GB, 122.2 tok/s), since only about 3B of their parameters activate per token. GPT-OSS 20B, a model OpenAI explicitly sized for a 16GB card, fits here with real headroom at Q6_K (19.54 GB, 60.5 tok/s). 70B-class dense models (Llama 3.3 70B, Qwen2.5 72B) don't fit a single card at any quantization at 8k context: even the smallest real builds, Q2_K at 32.88-33.73 GB, need CPU offload and crawl at 3.0-3.3 tok/s. This is exactly where a second RTX 3090 changes the picture: this site's calculator pools two cards at roughly 43.2 GB of usable VRAM, and Llama 3.3 70B fits there natively at Q3_K_M (40.72 GB, 15.1 tok/s), a roughly 9x jump over the single-card offload result of 1.7 tok/s at that same quant. The more commonly distributed Q4_K_M build doesn't quite clear that pooled 43.2 GB ceiling either (50.75 GB total), so it still spills a few GB into system RAM even across two cards, landing at 6.1 tok/s rather than running fully native; Q3_K_M, not Q4_K_M, is the honest answer for what two RTX 3090s actually run natively at 70B.

Full CUDA support under compute capability 8.6, the same mature Ampere target every RTX 30-series card shares: llama.cpp, Ollama, and every major inference tool has had five years to optimize for it, with none of Blackwell's early sm_120 driver caveats. Ampere predates Ada's hardware FP8 support and Blackwell's NVFP4, so GGUF's K-quants (down to Q2_K) are the only quantization path this site's calculator applies here; there's no newer low-bit format waiting to be unlocked the way there is on RTX 40- and 50-series cards. On the NVLink question specifically: llama.cpp's default multi-GPU behavior is pipeline/layer-split, where only one GPU computes at a time and the data crossing between cards each token is a small activation vector, not the whole model, so this site's calculator applies the identical multi-GPU bandwidth model whether or not an NVLink bridge is physically installed, matching this site's own multi-GPU guide's position that the bridge buys little decode speed for that common case. The bridge's real value is pooling capacity to roughly 48GB, not raw tokens per second, and even that takes real setup work: user reports on NVIDIA's own developer forums describe needing an SLI-certified motherboard, matched PCIe lane widths, and current drivers before peer-to-peer transfers reliably engage, friction that a bridge-free PCIe layer-split pair, which needs none of that, avoids entirely.