How much VRAM does the NVIDIA RTX A6000 have?

The NVIDIA RTX A6000 has 48 GB of GDDR6 with 768 GB/s memory bandwidth.

What is the NVIDIA RTX A6000 best for?

With 48 GB of VRAM, the NVIDIA RTX A6000 is ideal for running 70B-class models at Q3-class quantization and large MoE models, a workstation sweet spot for local inference.

What LLMs can the NVIDIA RTX A6000 run locally?

The NVIDIA RTX A6000 can run 62 of the 107 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Qwen 3.8 27B at Q8_0, Ornith 1.5 35B-A3B (MoE) at Q8_0, Ornith 1.5 9B at BF16.

Can the NVIDIA RTX A6000 run Gemma 4 31B?

Yes. The NVIDIA RTX A6000 runs Gemma 4 31B natively in VRAM at Q8_0 quantization, achieving approximately 18.5 tokens per second.

Can the NVIDIA RTX A6000 run Qwen 3.6 27B?

Yes. The NVIDIA RTX A6000 runs Qwen 3.6 27B natively in VRAM at Q8_0 quantization, achieving approximately 21.2 tokens per second.

Can the NVIDIA RTX A6000 run Qwen3 8B?

Yes. The NVIDIA RTX A6000 runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 36.2 tokens per second.

Can I rent the NVIDIA RTX A6000 instead of buying it?

Yes: RunPod and similar cloud GPU providers let you rent NVIDIA RTX A6000 instances by the hour, with no long-term contract. Rentals start from $0.33/hr on RunPod (as of Oct 17, 2026). This is often cheaper than buying if you only need it occasionally, and lets you try the GPU before committing to a purchase.

NVIDIA RTX A6000

The NVIDIA RTX A6000 has 48 GB VRAM and 768 GB/s memory bandwidth. It can run 62 of our 107 tracked models natively in VRAM at 8k context.

With 48 GB GDDR6, the NVIDIA RTX A6000 is a workstation-tier GPU that can run 62 models natively.

The NVIDIA RTX A6000 is an Ampere-generation workstation GPU with 48GB of ECC GDDR6 VRAM. Released in 2020, it bridges consumer RTX cards and data center A100s, running full CUDA drivers, supporting NVLink for multi-GPU setups, and fitting inside standard workstation towers. Its 48GB capacity lets it run 34B models at Q4 and most 70B models with CPU offload, making it a long-running workhorse for on-prem AI labs.

NVIDIA RTX A6000: 2020 Ampere workstation with 48GB GDDR6 at 768 GB/s. Bridges consumer and datacenter.

70B at Q3_K_M native (Q4_K_M's ~51GB total is just past this card's effective capacity). ~25-35 t/s for 7B, ~17 t/s for 70B at Q3_K_M.

Full CUDA with ECC memory. NVLink support for multi-GPU. Long-running workhorse for on-prem AI labs.

Vendor	NVIDIA
Architecture	Ampere
VRAM	48 GB
Memory type	GDDR6
Memory bandwidth	768 GB/s
Compute backend	CUDA
Tier	Workstation
Released	2020
Models (native)	62 / 107
Models (offload)	10 / 107

Software: Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.

Cloud GPU Rental

Don't want to buy an NVIDIA RTX A6000? RunPod is a cloud GPU rental service: rent one by the hour instead, no contract, no upfront hardware cost.

Pay by the hour · no contract · pods start in about a minute.

Rent an RTX A6000 from $0.33/hr on RunPod ↗ (+$5 signup credit)

Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing.

Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Popular models for this GPU

Qwen 2.5 72B Instruct Llama 3.3 70B Instruct DeepSeek R1 Distill Llama 70B Llama 3.1 70B Instruct Mixtral 8x7B Instruct v0.1

Models this GPU runs natively in VRAM (62)

Show 57 more

Models that fit with CPU offload (10)

These use system RAM for layers that don't fit in VRAM, so expect much slower inference.

Too large for this GPU (35)

Compare NVIDIA RTX A6000 with other GPUs

NVIDIA RTX A6000vsNVIDIA RTX 6000 Ada48 GB each

Continue reading

hardware11 min

Best GPU for Running LLMs Locally in 2026

vram-guides10 min

Best LLMs for 48 GB VRAM (2026)

hardware10 min

Best GPUs for Coding Agents in 2026

hardware14 min

Multi-GPU Setups for Local LLMs (2026): Split Modes & NVLink

hardware10 min

Best GPUs $1000+ for LLMs (2026)

hardware10 min

Used Enterprise GPUs for LLMs: A6000, A100, and Beyond

Frequently asked questions

How much VRAM does the NVIDIA RTX A6000 have?: The NVIDIA RTX A6000 has 48 GB of GDDR6 with 768 GB/s memory bandwidth.
What is the NVIDIA RTX A6000 best for?: With 48 GB of VRAM, the NVIDIA RTX A6000 is ideal for running 70B-class models at Q3-class quantization and large MoE models, a workstation sweet spot for local inference.
What LLMs can the NVIDIA RTX A6000 run locally?: The NVIDIA RTX A6000 can run 62 of the 107 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Qwen 3.8 27B at Q8_0, Ornith 1.5 35B-A3B (MoE) at Q8_0, Ornith 1.5 9B at BF16.
Can the NVIDIA RTX A6000 run Gemma 4 31B?: Yes. The NVIDIA RTX A6000 runs Gemma 4 31B natively in VRAM at Q8_0 quantization, achieving approximately 18.5 tokens per second.
Can the NVIDIA RTX A6000 run Qwen 3.6 27B?: Yes. The NVIDIA RTX A6000 runs Qwen 3.6 27B natively in VRAM at Q8_0 quantization, achieving approximately 21.2 tokens per second.
Can the NVIDIA RTX A6000 run Qwen3 8B?: Yes. The NVIDIA RTX A6000 runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 36.2 tokens per second.
Can I rent the NVIDIA RTX A6000 instead of buying it?: Yes: RunPod and similar cloud GPU providers let you rent NVIDIA RTX A6000 instances by the hour, with no long-term contract. Rentals start from $0.33/hr on RunPod (as of Oct 17, 2026). This is often cheaper than buying if you only need it occasionally, and lets you try the GPU before committing to a purchase.