CanItRun Logocanitrun.

NVIDIA RTX Pro 6000

The NVIDIA RTX Pro 6000 has 96 GB VRAM and 1344 GB/s memory bandwidth. It can run 67 of our 94 tracked models natively in VRAM at 8k context.

With 96 GB GDDR7, the NVIDIA RTX Pro 6000 is a workstation-tier GPU that can run 67 models natively. It runs 70B-class dense models and large MoE models entirely in VRAM.

The NVIDIA RTX Pro 6000 is the flagship Blackwell workstation GPU, doubling the RTX 6000 Ada's VRAM to 96GB of ECC GDDR7 on a 384-bit bus at 1,344 GB/s. It uses the full GB202 die with 24,576 CUDA cores (the same silicon as the RTX 5090) but in a workstation form factor with professional drivers, NVLink support, and error-correcting memory. The 96GB capacity is large enough to run 70B models at Q4_K_M or Q8_0 entirely in VRAM without any CPU offloading, and comfortably holds multiple models simultaneously. At ~$6,300 MSRP, it is the definitive single-GPU option for on-prem LLM inference when model fit and professional reliability matter more than cost.

NVIDIA RTX Pro 6000: 2025 Blackwell flagship workstation GPU built on the full GB202 die (the same silicon as the RTX 5090) with 96GB of ECC GDDR7, double the RTX 6000 Ada's capacity, at ~$6,300 MSRP.

70B models fit entirely in VRAM at Q4_K_M or even Q8_0 with no CPU offload, and it comfortably holds multiple smaller models loaded simultaneously. ~35-50 t/s for 7B Q4.

Full CUDA with ECC and NVLink support. The definitive single-GPU option for on-prem LLM inference when model fit matters more than cost; no consumer card at any price offers this much VRAM in one slot.

VendorNVIDIA
ArchitectureBlackwell
VRAM96 GB
Memory typeGDDR7
Memory bandwidth1344 GB/s
Compute backendCUDA
TierWorkstation
Released2025
Models (native)67 / 94
Models (offload)4 / 94
Software: Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

Cloud GPU Rental

Don't want to buy a NVIDIA RTX Pro 6000? RunPod is a cloud GPU rental service: rent one by the hour instead, no contract, no upfront hardware cost.

Pay by the hour · no contract · pods start in about a minute.

Rent a NVIDIA RTX Pro 6000 on RunPod ↗ (+$5 signup credit)

Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Popular models for this GPU

Step 3.7 FlashStep 3.5 FlashMixtral 8x22B Instruct v0.1Mistral Medium 3.5 128BQwen 3.5 122B-A10B (MoE)

Models this GPU runs natively in VRAM (67)

Show 62 more

Models that fit with CPU offload (4)

These use system RAM for layers that don't fit in VRAM, so expect much slower inference.

Too large for this GPU (23)

Compare NVIDIA RTX Pro 6000 with other GPUs

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

How much VRAM does the NVIDIA RTX Pro 6000 have?
The NVIDIA RTX Pro 6000 has 96 GB of GDDR7 with 1344 GB/s memory bandwidth.
What is the NVIDIA RTX Pro 6000 best for?
With 96 GB of VRAM, the NVIDIA RTX Pro 6000 is a high-capacity workstation platform that runs 70B-class dense models and large MoE models natively, with plenty of room for long context.
What LLMs can the NVIDIA RTX Pro 6000 run locally?
The NVIDIA RTX Pro 6000 can run 67 of the 94 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Mistral Medium 3.5 128B at NVFP4, Step 3.7 Flash at Q2_K, Qwen 3.8 27B at BF16.
Can the NVIDIA RTX Pro 6000 run Gemma 4 31B?
Yes. The NVIDIA RTX Pro 6000 runs Gemma 4 31B natively in VRAM at BF16 quantization, achieving approximately 13.9 tokens per second.
Can the NVIDIA RTX Pro 6000 run Qwen 3.6 27B?
Yes. The NVIDIA RTX Pro 6000 runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 16 tokens per second.
Can the NVIDIA RTX Pro 6000 run Qwen3 8B?
Yes. The NVIDIA RTX Pro 6000 runs Qwen3 8B natively in VRAM at FP32 quantization, achieving approximately 26.3 tokens per second.