NVIDIA RTX 4090 vs NVIDIA RTX 4080
Side-by-side local AI comparison: VRAM, memory bandwidth, model compatibility, and estimated tokens per second across 94 open-weight models.
Quick verdict
NVIDIA RTX 4090 wins for local AI inference. It has 8 GB more VRAM and 41% more memory bandwidth, runs 52 models natively (vs 45), and exclusively fits 7 models the other cannot.
Analysis
The RTX 4090 and RTX 4080 launched five weeks apart in late 2022 as Ada Lovelace's two highest consumer tiers, at $1,599 and $1,199 respectively. For local LLM inference, the 33% price gap doesn't buy a proportional capability gap: it buys 50% more VRAM (24GB vs 16GB) and 40.6% more bandwidth (1,008 vs 717 GB/s), and that combination pushes the RTX 4090 past a real capability line the RTX 4080 can't cross at all.
Both cards share the same 4th-gen Tensor Core generation, so the difference comes down to the memory subsystem: the RTX 4090's near-uncut AD102 die carries 24GB of GDDR6X on a 384-bit bus, against the RTX 4080's cut-down AD103 die on a 256-bit bus. For models both cards fit, that 40.6% bandwidth gap shows up almost exactly in this site's calculator: Llama 3.1 8B at its recommended Q5_K_M decodes at 96.8 tok/s on the RTX 4090 versus 68.8 tok/s on the RTX 4080, a 40.7% gap, and GPT-OSS 20B at its recommended Q4_K_M decodes at 87.2 tok/s versus 62 tok/s, a near-identical 40.6% gap. The real difference is what the extra 8GB unlocks rather than just what it speeds up: Qwen 3.6 27B fits the RTX 4090 natively at its recommended Q4_K_M (19.02 GB, 38.6 tok/s), while the RTX 4080's 16GB can't hold that same build and drops to a CPU-offloaded 9.6 tok/s, forcing a step down to Q3_K_M (15.15 GB, 34.5 tok/s) just to stay native. Qwen3 32B draws an even harder line: it fits the RTX 4090 natively at Q3_K_M (19.17 GB, 38.3 tok/s), but doesn't fit the RTX 4080 at any quantization on this site's standard ladder, landing at 9.1 tok/s offloaded there instead.
Bottom line: For anyone whose models fit both cards' VRAM, the RTX 4090 is a straightforward roughly 40% speed premium tracking its bandwidth advantage almost exactly, worth it only if that 33% price gap is worth 40% more tok/s. The real decision point is 27-32B-class dense models: the RTX 4090 runs Qwen 3.6 27B and Qwen3 32B natively where the RTX 4080 either has to drop a quant level or fall back to CPU offload entirely. Anyone planning to run models in that range should treat the RTX 4090's extra 8GB, not its extra bandwidth, as the deciding factor.
Specs comparison
| Spec | NVIDIA RTX 4090 | NVIDIA RTX 4080 |
|---|---|---|
| VRAM | 24 GB | 16 GB |
| Memory type | GDDR6X | GDDR6X |
| Bandwidth | 1008 GB/s(+41%) | 717 GB/s |
| Architecture | Ada Lovelace | Ada Lovelace |
| Backend | CUDA | CUDA |
| Tier | Consumer | Consumer |
| Released | 2022 | 2022 |
| Models (native) | 52 | 45 |
Estimated tokens per second
Computed from memory bandwidth and model active-parameter weight. Assumes model fits natively in VRAM.
| Model | NVIDIA RTX 4090 | NVIDIA RTX 4080 | Delta |
|---|---|---|---|
| Llama 3.3 70B Instruct(70B) | N/A | N/A | N/A |
| Qwen 3.6 27B(27B) | 33.2 t/s(Q5_K_M) | 34.5 t/s(Q3_K_M) | -4% |
| Llama 3.1 8B Instruct(8B) | 38.4 t/s(BF16) | 48.7 t/s(Q8_0) | -21% |
| Qwen 2.5 7B Instruct(7.6B) | 41.8 t/s(BF16) | 54.5 t/s(Q8_0) | -23% |
Delta is NVIDIA RTX 4090 relative to NVIDIA RTX 4080.
Only NVIDIA RTX 4090 can run(7)
Only NVIDIA RTX 4080 can run(0)
No exclusive models: NVIDIA RTX 4090 can run everything NVIDIA RTX 4080 can.
Both run natively(45)
These models fit in VRAM on both GPUs. Bandwidth determines which runs them faster.
- Qwen 3.5 35B-A3B (MoE)131.6 t/svs117.2 t/s
- Ornith 1.5 35B-A3B (MoE)131.6 t/svs117.2 t/s
- Nemotron 3 Nano 30B100.4 t/svs109.8 t/s
- Gemma 4 31B32.4 t/svs35.3 t/s
- Qwen3 30B-A3B (MoE)95 t/svs101 t/s
- Nemotron 3.5 Lightning 30B-A3B106.7 t/svs120.7 t/s
- Muse Glimmer 30B32.8 t/svs34.4 t/s
- Gemma 2 27B Instruct33.3 t/svs34.7 t/s
- Gemma 3 27B Instruct36.4 t/svs39.4 t/s
- Qwen 3.6 27B33.2 t/svs34.5 t/s
- UI-Mate 27B33.2 t/svs34.5 t/s
- Bonsai 27B74.4 t/svs52.9 t/s
- Qwen 3.8 27B33.2 t/svs34.5 t/s
- Gemma 4 26B (MoE)69.7 t/svs72 t/s
- Mistral Small 3.1 24B Instruct35.6 t/svs36.2 t/s
- Mistral Small 22B32.6 t/svs37.1 t/s
- +29 more on both
Which should you choose?
- • You need to run larger models (>16 GB VRAM)
- • Faster token generation is the priority
Frequently asked questions
- Which is better for local AI, the NVIDIA RTX 4090 or NVIDIA RTX 4080?
- For local AI inference, the NVIDIA RTX 4090 has the edge. It offers 24 GB VRAM (vs 16 GB) and 1008 GB/s bandwidth (vs 717 GB/s), letting it run 52 models natively in VRAM vs 45 for its rival.
- How much VRAM does the NVIDIA RTX 4090 have vs the NVIDIA RTX 4080?
- The NVIDIA RTX 4090 has 24 GB of GDDR6X at 1008 GB/s. The NVIDIA RTX 4080 has 16 GB of GDDR6X at 717 GB/s. The NVIDIA RTX 4090 has 8 GB more VRAM, allowing it to run 7 models the NVIDIA RTX 4080 cannot fit natively.
- Can the NVIDIA RTX 4090 run Llama 3.3 70B?
- The NVIDIA RTX 4090 can run Llama 3.3 70B with CPU offload at Q3_K_M, but at reduced speed.
- Can the NVIDIA RTX 4080 run Llama 3.3 70B?
- The NVIDIA RTX 4080 can run Llama 3.3 70B with CPU offload at Q3_K_M, but at reduced speed.
- What is the difference between the NVIDIA RTX 4090 and NVIDIA RTX 4080 for AI?
- The key difference for AI inference is VRAM and memory bandwidth. The NVIDIA RTX 4090 has 24 GB VRAM at 1008 GB/s (CUDA backend). The NVIDIA RTX 4080 has 16 GB VRAM at 717 GB/s (CUDA backend). VRAM determines which models fit; bandwidth determines tokens per second. The NVIDIA RTX 4090 runs 52 models natively vs 45 for the NVIDIA RTX 4080.