CanItRun Logocanitrun.

NVIDIA RTX Pro 6000

The NVIDIA RTX Pro 6000 has 96 GB VRAM and 1344 GB/s memory bandwidth. It can run 57 of our 70 tracked models natively in VRAM at 8k context.

The NVIDIA RTX Pro 6000 is the flagship Blackwell workstation GPU, doubling the RTX 6000 Ada's VRAM to 96GB of ECC GDDR7 on a 384-bit bus at 1,344 GB/s. It uses the full GB202 die with 24,576 CUDA cores — the same silicon as the RTX 5090 — but in a workstation form factor with professional drivers, NVLink support, and error-correcting memory. The 96GB capacity is large enough to run 70B models at Q4_K_M or Q8_0 entirely in VRAM without any CPU offloading, and comfortably holds multiple models simultaneously. At ~$6,300 MSRP, it is the definitive single-GPU option for on-prem LLM inference when model fit and professional reliability matter more than cost.

The NVIDIA RTX Pro 6000 is a professional workstation NVIDIA GPU based on the Blackwell architecture. Released in 2025. It features 96 GB of GDDR7 at 1344 GB/s memory bandwidth. Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

For local LLM inference, this GPU runs 57 of the 70 models we track natively in VRAM at 8K context. The largest model it handles in VRAM is Qwen3 235B-A22B (MoE) (204.3 t/s at Q2_K). It can run all tracked models including 405B-class frontier models entirely in VRAM. On Llama 3.3 70B Instruct, it achieves approximately 38.4 tokens per second at NVFP4 quantization. An additional 1 models fit with CPU offload — slower but usable.

NVIDIA's CUDA ecosystem provides broad out-of-the-box support across llama.cpp, Ollama, vLLM, and TensorRT-LLM. Among workstation GPUs, it sits above Apple M4 Max (96GB) and NVIDIA RTX 6000 Ada in performance, but below NVIDIA A100 40GB.

VendorNVIDIA
ArchitectureBlackwell
VRAM96 GB
Memory typeGDDR7
Memory bandwidth1344 GB/s
Compute backendCUDA
TierWorkstation
Released2025
Models (native)57 / 70
Models (offload)1 / 70
Software: Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

Models this GPU runs natively in VRAM (57)

Models that fit with CPU offload (1)

These use system RAM for layers that don't fit in VRAM — expect much slower inference.

Too large for this GPU (12)

Compare NVIDIA RTX Pro 6000 with other GPUs

Frequently asked questions

How much VRAM does the NVIDIA RTX Pro 6000 have?
The NVIDIA RTX Pro 6000 has 96 GB of GDDR7 with 1344 GB/s memory bandwidth.
What LLMs can the NVIDIA RTX Pro 6000 run locally?
The NVIDIA RTX Pro 6000 can run 57 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.3 70B Instruct at NVFP4, Llama 3.1 8B Instruct at FP32, Llama 3.2 3B Instruct at FP32.
Can the NVIDIA RTX Pro 6000 run Llama 3.3 70B Instruct?
Yes. The NVIDIA RTX Pro 6000 runs Llama 3.3 70B Instruct natively in VRAM at NVFP4 quantization, achieving approximately 38.4 tokens per second.
Can the NVIDIA RTX Pro 6000 run Qwen 3.6 27B?
Yes. The NVIDIA RTX Pro 6000 runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 24.9 tokens per second.
Can the NVIDIA RTX Pro 6000 run Llama 3.1 8B Instruct?
Yes. The NVIDIA RTX Pro 6000 runs Llama 3.1 8B Instruct natively in VRAM at FP32 quantization, achieving approximately 42 tokens per second.