How much VRAM does the NVIDIA RTX 3090 have?

The NVIDIA RTX 3090 has 24 GB of GDDR6X with 936 GB/s memory bandwidth.

What is the NVIDIA RTX 3090 best for?

With 24 GB of VRAM, the NVIDIA RTX 3090 is well-suited for running 7B–32B models at Q4 with room for context, making it a great all-rounder for local LLM inference.

What LLMs can the NVIDIA RTX 3090 run locally?

The NVIDIA RTX 3090 can run 42 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.1 8B Instruct at BF16, Llama 3.2 3B Instruct at FP32, Llama 3.2 1B Instruct at FP32.

Can the NVIDIA RTX 3090 run Llama 3.3 70B Instruct?

The NVIDIA RTX 3090 can run Llama 3.3 70B Instruct with CPU offload at NVFP4 quantization, but inference will be slower than native VRAM execution.

Can the NVIDIA RTX 3090 run Qwen 3.6 27B?

Yes. The NVIDIA RTX 3090 runs Qwen 3.6 27B natively in VRAM at NVFP4 quantization, achieving approximately 69.3 tokens per second.

Can the NVIDIA RTX 3090 run Llama 3.1 8B Instruct?

Yes. The NVIDIA RTX 3090 runs Llama 3.1 8B Instruct natively in VRAM at BF16 quantization, achieving approximately 58.5 tokens per second.

NVIDIA RTX 3090

The NVIDIA RTX 3090 has 24 GB VRAM and 936 GB/s memory bandwidth. It can run 42 of our 70 tracked models natively in VRAM at 8k context.

With 24 GB GDDR6X, the NVIDIA RTX 3090 is a consumer-tier GPU that can run 42 models natively. It handles 70B-class models at Q4 quantization.

NVIDIA RTX 3090: September 2020 with 24GB GDDR6X at 936 GB/s — the used market king for local LLM.

7B-32B at Q4 native. 70B with CPU offload. ~10-18 t/s for 7B, ~5-8 t/s for 32B.

Full CUDA support. Best value 24GB GPU on used market. NVLink enables dual-3090 for 70B native.

Vendor	NVIDIA
Architecture	Ampere
VRAM	24 GB
Memory type	GDDR6X
Memory bandwidth	936 GB/s
Compute backend	CUDA
Tier	Consumer
Released	2020
Models (native)	42 / 70
Models (offload)	11 / 70

Software: Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

Popular models for this GPU

Mixtral 8x7B Instruct v0.1 Qwen 3.5 35B-A3B (MoE)Qwen 3.6 35B Yi 1.5 34B Chat Qwen3 32B

Models this GPU runs natively in VRAM (42)

Models that fit with CPU offload (11)

These use system RAM for layers that don't fit in VRAM — expect much slower inference.

Too large for this GPU (17)

Compare NVIDIA RTX 3090 with other GPUs

Frequently asked questions

How much VRAM does the NVIDIA RTX 3090 have?: The NVIDIA RTX 3090 has 24 GB of GDDR6X with 936 GB/s memory bandwidth.
What is the NVIDIA RTX 3090 best for?: With 24 GB of VRAM, the NVIDIA RTX 3090 is well-suited for running 7B–32B models at Q4 with room for context, making it a great all-rounder for local LLM inference.
What LLMs can the NVIDIA RTX 3090 run locally?: The NVIDIA RTX 3090 can run 42 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.1 8B Instruct at BF16, Llama 3.2 3B Instruct at FP32, Llama 3.2 1B Instruct at FP32.
Can the NVIDIA RTX 3090 run Llama 3.3 70B Instruct?: The NVIDIA RTX 3090 can run Llama 3.3 70B Instruct with CPU offload at NVFP4 quantization, but inference will be slower than native VRAM execution.
Can the NVIDIA RTX 3090 run Qwen 3.6 27B?: Yes. The NVIDIA RTX 3090 runs Qwen 3.6 27B natively in VRAM at NVFP4 quantization, achieving approximately 69.3 tokens per second.
Can the NVIDIA RTX 3090 run Llama 3.1 8B Instruct?: Yes. The NVIDIA RTX 3090 runs Llama 3.1 8B Instruct natively in VRAM at BF16 quantization, achieving approximately 58.5 tokens per second.