How much VRAM does the NVIDIA RTX 5070 have?

The NVIDIA RTX 5070 has 12 GB of GDDR7 with 672 GB/s memory bandwidth.

What LLMs can the NVIDIA RTX 5070 run locally?

The NVIDIA RTX 5070 can run 28 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.1 8B Instruct at NVFP4, Llama 3.2 3B Instruct at BF16, Llama 3.2 1B Instruct at FP32.

Can the NVIDIA RTX 5070 run Llama 3.3 70B Instruct?

The NVIDIA RTX 5070 can run Llama 3.3 70B Instruct with CPU offload at Q3_K_M quantization, but inference will be slower than native VRAM execution.

Can the NVIDIA RTX 5070 run Qwen 3.6 27B?

The NVIDIA RTX 5070 can run Qwen 3.6 27B with CPU offload at NVFP4 quantization, but inference will be slower than native VRAM execution.

Can the NVIDIA RTX 5070 run Llama 3.1 8B Instruct?

Yes. The NVIDIA RTX 5070 runs Llama 3.1 8B Instruct natively in VRAM at NVFP4 quantization, achieving approximately 168 tokens per second.

NVIDIA RTX 5070

The NVIDIA RTX 5070 has 12 GB VRAM and 672 GB/s memory bandwidth. It can run 28 of our 70 tracked models natively in VRAM at 8k context.

The NVIDIA RTX 5070 features 12GB GDDR7 on a 192-bit bus (672 GB/s) with 6,144 CUDA cores. It handles 7B-class models at Q4_K_M in VRAM, but 14B+ models require CPU offload. The 12GB capacity is a step down from the 5070 Ti's 16GB, making it a 1080p–1440p gaming card first and an entry-level AI GPU second.

The NVIDIA RTX 5070 is a consumer-grade NVIDIA GPU based on the Blackwell architecture. Released in 2025. It features 12 GB of GDDR7 at 672 GB/s memory bandwidth. Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

For local LLM inference, this GPU runs 28 of the 70 models we track natively in VRAM at 8K context. The largest model it handles in VRAM is Gemma 4 26B (MoE) (591.3 t/s at Q2_K). Its VRAM limits it to smaller models (1-8B parameters), which makes it suitable for prototyping and edge inference. On Llama 3.1 8B Instruct, it achieves approximately 168 tokens per second at NVFP4 quantization. An additional 19 models fit with CPU offload — slower but usable.

NVIDIA's CUDA ecosystem provides broad out-of-the-box support across llama.cpp, Ollama, vLLM, and TensorRT-LLM. Among consumer GPUs, it sits above AMD Radeon RX 7900 GRE and Intel Arc A770 16GB in performance, but below NVIDIA RTX 4080.

Vendor	NVIDIA
Architecture	Blackwell
VRAM	12 GB
Memory type	GDDR7
Memory bandwidth	672 GB/s
Compute backend	CUDA
Tier	Consumer
Released	2025
Models (native)	28 / 70
Models (offload)	19 / 70

Software: Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

Models this GPU runs natively in VRAM (28)

Models that fit with CPU offload (19)

These use system RAM for layers that don't fit in VRAM — expect much slower inference.

Too large for this GPU (23)

Frequently asked questions

How much VRAM does the NVIDIA RTX 5070 have?: The NVIDIA RTX 5070 has 12 GB of GDDR7 with 672 GB/s memory bandwidth.
What LLMs can the NVIDIA RTX 5070 run locally?: The NVIDIA RTX 5070 can run 28 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.1 8B Instruct at NVFP4, Llama 3.2 3B Instruct at BF16, Llama 3.2 1B Instruct at FP32.
Can the NVIDIA RTX 5070 run Llama 3.3 70B Instruct?: The NVIDIA RTX 5070 can run Llama 3.3 70B Instruct with CPU offload at Q3_K_M quantization, but inference will be slower than native VRAM execution.
Can the NVIDIA RTX 5070 run Qwen 3.6 27B?: The NVIDIA RTX 5070 can run Qwen 3.6 27B with CPU offload at NVFP4 quantization, but inference will be slower than native VRAM execution.
Can the NVIDIA RTX 5070 run Llama 3.1 8B Instruct?: Yes. The NVIDIA RTX 5070 runs Llama 3.1 8B Instruct natively in VRAM at NVFP4 quantization, achieving approximately 168 tokens per second.