One A6000 or two RTX 3090s?

Dual 3090s (~$1500-1800) give same 48 GB at roughly half the cost of used A6000 ($3000-4000). A6000 wins on simplicity, power (300W vs 700W), ECC. For hobbyists, dual 3090s are better value.

Can I mix different GPU models?

Technically yes via llama.cpp tensor split, but effective speed limited by slowest card. Use larger-VRAM card as --main-gpu.

System RAM needed with 48 GB VRAM?

At least 64 GB, preferably 128 GB. System RAM used for model loading and overflow. 32 GB is absolute minimum — you will hit swap during loading.

Apple Silicon vs dual 3090s for 48 GB?

Dual 3090s: 15-22 tok/s on 70B Q4. M4 Pro: 12-15 tok/s but ~50W vs 700W, near-silent. Speed vs simplicity trade-off.

Best LLMs for 48 GB VRAM (2026)

CanItRunJuly 1, 20269 min readVRAM Guides

48 GB: The 70B Gateway

48 GB is the threshold where the 70B class becomes your daily driver. Three paths: dual consumer GPUs (2x RTX 3090 = 48 GB, ~$1500-1800 used), single workstation GPU (RTX A6000 48GB, ~$3000-4000 used), or Apple Silicon (M4 Pro 48GB, $2399). Dual 3090s are most popular — lowest cost, and llama.cpp tensor parallelism makes multi-GPU straightforward. Key: multi-GPU scaling is sublinear — ~80% usable VRAM, ~1.5x speed (not 2x). NVLink not needed for inference.

70B at Q4: The Daily Driver

Llama 3.3 70B at Q4_K_M (~39 GB) is the defining model. On dual RTX 3090s: 15-22 tok/s at 8K context. Llama 3.3 matches Llama 3.1 405B on many benchmarks at one-fifth the size. Qwen2.5 72B at Q4 (~44 GB) is a strong multilingual alternative. DeepSeek R1 Distill Llama 70B at Q4 (~40-43 GB) adds chain-of-thought — but generates 2-5x more tokens per query.

2x RTX 3090 in Practice

Requirements: motherboard with 2 PCIe x16 slots, 1000W+ PSU (dual 3090s draw ~700W), adequate airflow. llama.cpp flags: --tensor-split and --split-mode row (~2 tok/s extra). Ollama auto-detects multi-GPU. LM Studio does NOT support multi-GPU. Set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 to allow system RAM fallback instead of OOM crash.

Single-GPU Options: A6000 and Mac

RTX A6000 48GB (used, $3000-4000): ECC, 768 GB/s, blower cooler, no multi-GPU complexity. M4 Pro 48GB (Mac mini, $2399): 70B Q4 at 12-15 tok/s, ~50W, near-silent. Unified memory eliminates multi-GPU complexity entirely. For simplicity and noise over raw speed, Mac is compelling.

KV Cache and Long Context

At FP16, 70B KV cache = ~2.5 MB/token. 8K context = ~20 GB cache. 32K context = ~80 GB — exceeds 48 GB! Solution: KV cache quantization. llama.cpp flags: --cache-type-k q8_0 --cache-type-v q8_0 (50-75% reduction). TurboQuant (3.5 bpw) reduces further. For 128K context on 70B, you need 64 GB+.

Beyond 70B: What 48 GB Cannot Do

Llama 3.1 405B Q4 needs ~230 GB. Llama 4 Scout (109B MoE) Q4 needs ~55 GB — slightly over. Mixtral 8x22B Q4 needs ~80 GB. Qwen3 235B-A22B Q4 needs ~140 GB. The 70-72B dense class is the ceiling. Good news: 70B models have improved so much (Llama 3.3 matching 405B) that you are not missing much.

Frequently asked questions

One A6000 or two RTX 3090s?: Dual 3090s (~$1500-1800) give same 48 GB at roughly half the cost of used A6000 ($3000-4000). A6000 wins on simplicity, power (300W vs 700W), ECC. For hobbyists, dual 3090s are better value.
Can I mix different GPU models?: Technically yes via llama.cpp tensor split, but effective speed limited by slowest card. Use larger-VRAM card as --main-gpu.
System RAM needed with 48 GB VRAM?: At least 64 GB, preferably 128 GB. System RAM used for model loading and overflow. 32 GB is absolute minimum — you will hit swap during loading.
Apple Silicon vs dual 3090s for 48 GB?: Dual 3090s: 15-22 tok/s on 70B Q4. M4 Pro: 12-15 tok/s but ~50W vs 700W, near-silent. Speed vs simplicity trade-off.