How much VRAM does the NVIDIA RTX Pro 6000 have?

The NVIDIA RTX Pro 6000 has 96 GB of GDDR7 with 1344 GB/s memory bandwidth.

What is the NVIDIA RTX Pro 6000 best for?

With 96 GB of VRAM, the NVIDIA RTX Pro 6000 is a server-class GPU designed for running the largest open-weight models (70B–405B) at high quantization with ample context.

What LLMs can the NVIDIA RTX Pro 6000 run locally?

The NVIDIA RTX Pro 6000 can run 57 of the 71 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.3 70B Instruct at NVFP4, Llama 3.1 8B Instruct at FP32, Llama 3.2 3B Instruct at FP32.

Can the NVIDIA RTX Pro 6000 run Llama 3.3 70B Instruct?

Yes. The NVIDIA RTX Pro 6000 runs Llama 3.3 70B Instruct natively in VRAM at NVFP4 quantization, achieving approximately 38.4 tokens per second.

Can the NVIDIA RTX Pro 6000 run Qwen 3.6 27B?

Yes. The NVIDIA RTX Pro 6000 runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 24.9 tokens per second.

Can the NVIDIA RTX Pro 6000 run Llama 3.1 8B Instruct?

Yes. The NVIDIA RTX Pro 6000 runs Llama 3.1 8B Instruct natively in VRAM at FP32 quantization, achieving approximately 42 tokens per second.

NVIDIA RTX Pro 6000

The NVIDIA RTX Pro 6000 has 96 GB VRAM and 1344 GB/s memory bandwidth. It can run 57 of our 71 tracked models natively in VRAM at 8k context.

With 96 GB GDDR7, the NVIDIA RTX Pro 6000 is a workstation-tier GPU that can run 57 models natively. It handles 70B-class models at Q4 quantization.

The NVIDIA RTX Pro 6000 is the flagship Blackwell workstation GPU, doubling the RTX 6000 Ada's VRAM to 96GB of ECC GDDR7 on a 384-bit bus at 1,344 GB/s. It uses the full GB202 die with 24,576 CUDA cores — the same silicon as the RTX 5090 — but in a workstation form factor with professional drivers, NVLink support, and error-correcting memory. The 96GB capacity is large enough to run 70B models at Q4_K_M or Q8_0 entirely in VRAM without any CPU offloading, and comfortably holds multiple models simultaneously. At ~$6,300 MSRP, it is the definitive single-GPU option for on-prem LLM inference when model fit and professional reliability matter more than cost.

The NVIDIA RTX Pro 6000 is a professional workstation NVIDIA GPU based on the Blackwell architecture. Released in 2025. It features 96 GB of GDDR7 at 1344 GB/s memory bandwidth. Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

For local LLM inference, this GPU runs 57 of the 71 models we track natively in VRAM at 8K context. The largest model it handles in VRAM is Qwen3 235B-A22B (MoE) (204.3 t/s at Q2_K). It can run all tracked models including 405B-class frontier models entirely in VRAM. On Llama 3.3 70B Instruct, it achieves approximately 38.4 tokens per second at NVFP4 quantization. An additional 1 models fit with CPU offload — slower but usable.

NVIDIA's CUDA ecosystem provides broad out-of-the-box support across llama.cpp, Ollama, vLLM, and TensorRT-LLM. Among workstation GPUs, it sits above Apple M3 Ultra (96GB) and Intel Data Center GPU Max 1100 in performance, but below NVIDIA A100 40GB.

Vendor	NVIDIA
Architecture	Blackwell
VRAM	96 GB
Memory type	GDDR7
Memory bandwidth	1344 GB/s
Compute backend	CUDA
Tier	Workstation
Released	2025
Models (native)	57 / 71
Models (offload)	1 / 71

Popular models for this GPU

Models this GPU runs natively in VRAM (57)

Models that fit with CPU offload (1)

Too large for this GPU (13)

Compare NVIDIA RTX Pro 6000 with other GPUs

Models mentioned

GPUs mentioned

Frequently asked questions