NVIDIA RTX 3060 12GB
The NVIDIA RTX 3060 12GB has 12 GB VRAM and 360 GB/s memory bandwidth. It can run 30 of our 97 tracked models natively in VRAM at 8k context.
With 12 GB GDDR6, the NVIDIA RTX 3060 12GB is a consumer-tier GPU that can run 30 models natively. This site's calculator puts Llama 3.1 8B at 30.6 tok/s at Q6_K (8.6 GB) and 39.4 tok/s at Q4_K_M (6.7 GB), both fit natively at 8k context. Qwen2.5 14B is the real ceiling: Q4_K_M (11.8 GB) actually spills into system RAM once overhead is counted, so Q3_K_M (9.7 GB, 27 tok/s) is the largest quant that stays fully in VRAM. 27B+ models need CPU offload at any practical quant.
The NVIDIA RTX 3060 12GB is a budget Ampere GPU with a surprisingly generous 12GB GDDR6 on a 192-bit bus at 360 GB/s. The high VRAM-to-price ratio made it a community favorite for local LLM experimentation, running 7B models at Q5–Q8 and 13B models at Q4 with some headroom. Bandwidth is the limiting factor for inference speed.
NVIDIA RTX 3060 12GB: Announced at CES on January 12, 2021 and on sale five weeks later (February 25) at a $329 MSRP, built on the Ampere GA106 die (3,584 CUDA cores, 170W TDP, PCIe 4.0 x16). Some 2022-production units shipped on a cut-down GA104-150 die instead during the chip shortage; VRAM, bus width, and bandwidth are identical either way.
This site's calculator puts Llama 3.1 8B at 30.6 tok/s at Q6_K (8.6 GB) and 39.4 tok/s at Q4_K_M (6.7 GB), both fit natively at 8k context. Qwen2.5 14B is the real ceiling: Q4_K_M (11.8 GB) actually spills into system RAM once overhead is counted, so Q3_K_M (9.7 GB, 27 tok/s) is the largest quant that stays fully in VRAM. 27B+ models need CPU offload at any practical quant.
Compute capability 8.6, full, mature CUDA support with no version caveats; every modern llama.cpp or Ollama build works out of the box. The 12GB capacity itself wasn't an AI-era design choice: NVIDIA's own 192-bit memory bus only splits cleanly into 6GB or 12GB configurations, so 12GB was the higher of two options in 2021, not a deliberate hobbyist play. That accident left it with more VRAM than the RTX 3060 Ti and RTX 3070 (8GB each), the RTX 3080 (10GB, a genuinely higher-tier card at the time), and even the RTX 4060 Ti's 8GB base config two years later, which is what actually made it a budget local-LLM favorite.
| Vendor | NVIDIA |
| Architecture | Ampere |
| VRAM | 12 GB |
| Memory type | GDDR6 |
| Memory bandwidth | 360 GB/s |
| Compute backend | CUDA |
| Tier | Consumer |
| Released | 2021 |
| Models (native) | 30 / 97 |
| Models (offload) | 27 / 97 |
More VRAM than three newer, faster cards
VRAM and bandwidth don't move together across NVIDIA's stack; a card can lead on one and trail on the other. Plotting this card against three cards released after it, all nominally a step up:
This card's 12GB beats the RTX 4060 (8GB) and RTX 5060 (8GB), one and two generations newer, and even the RTX 3080 (10GB), a genuinely higher-tier card from the same generation. Bandwidth tells a mixed story: at 360 GB/s it trails the RTX 3080 (760) and both Blackwell 50-series cards here (448 each), but it's actually faster than the RTX 4060 and RTX 4060 Ti 16GB (272 and 288 GB/s); Ada's 8-16GB tier cut bandwidth harder than the Ampere or Blackwell generations on either side of it. For a local LLM, VRAM decides what fits; bandwidth decides how fast it runs once it does; this card was built lopsided toward the first.
Popular models for this GPU
Models this GPU runs natively in VRAM (30)
- Bonsai 27B27B · MMLU-Pro 81.5Ternary (Q2_0) · ~26.6 t/s
- Gemma 4 26B (MoE)25.2B · MMLU-Pro 82.6Q2_K · ~45 t/s
- Qwen3 14B14.8B · MMLU-Pro 61.0Q3_K_M · ~27.7 t/s
- Qwen 2.5 14B Instruct14.7B · MMLU-Pro 63.7Q3_K_M · ~27 t/s
- Phi-4 14B Instruct14B · MMLU-Pro 70.4Q4_K_M · ~23.7 t/s
Show 25 more
- Mistral Nemo 12B Instruct12.2B · MMLU-Pro 35.6Q5_K_M · ~23.3 t/s
- Gemma 3 12B Instruct12.2B · MMLU-Pro 60.6Q5_K_M · ~24 t/s
- Gemma 4 12B (Unified)12B · MMLU-Pro 77.2Q3_K_M · ~26 t/s
- Gemma 2 9B Instruct9.2B · MMLU-Pro 32.0Q5_K_M · ~25 t/s
- Qwen 3.5 9B9B · MMLU-Pro 82.5Q8_0 · ~23.8 t/s
- Ornith 1.5 9B9B · MMLU-Pro N/AQ8_0 · ~23.8 t/s
- Llama 3.1 8B Instruct8B · MMLU-Pro 48.3Q8_0 · ~24.4 t/s
- DeepSeek R1 Distill Llama 8B8B · MMLU-Pro 41.0Q8_0 · ~24.4 t/s
- Qwen3 8B8B · MMLU-Pro 56.7Q8_0 · ~24.1 t/s
- Qwen 2.5 7B Instruct7.6B · MMLU-Pro 56.3Q8_0 · ~27.4 t/s
- Mistral 7B Instruct v0.37.25B · MMLU-Pro 30.0Q8_0 · ~26.6 t/s
- Gemma 3 4B Instruct4B · MMLU-Pro 43.6BF16 · ~27.5 t/s
- Gemma 4 E4B4B · MMLU-Pro 69.4BF16 · ~26 t/s
- Phi-3.5 Mini Instruct3.8B · MMLU-Pro 47.4Q8_0 · ~32.2 t/s
- Phi-4-mini Instruct3.8B · MMLU-Pro 67.3BF16 · ~27 t/s
- Llama 3.2 3B Instruct3.2B · MMLU-Pro 24.0BF16 · ~31.9 t/s
- Qwen 2.5 3B Instruct3.1B · MMLU-Pro 32.4BF16 · ~36 t/s
- Gemma 2 2B Instruct2.6B · MMLU-Pro 17.8BF16 · ~38.5 t/s
- Gemma 4 E2B2B · MMLU-Pro 60.0BF16 · ~53.1 t/s
- SmolLM2 1.7B Instruct1.7B · MMLU-Pro 19.0BF16 · ~46.7 t/s
- Qwen 2.5 1.5B Instruct1.5B · MMLU-Pro 16.8BF16 · ~72.3 t/s
- Llama 3.2 1B Instruct1.24B · MMLU-Pro 12.5BF16 · ~85.1 t/s
- Gemma 3 1B Instruct1B · MMLU-Pro 14.7BF16 · ~100.6 t/s
- Qwen 2.5 0.5B Instruct0.5B · MMLU-Pro 10.0BF16 · ~212.6 t/s
- SmolLM2 360M Instruct0.36B · MMLU-Pro 8.0BF16 · ~221.7 t/s
Models that fit with CPU offload (27)
These use system RAM for layers that don't fit in VRAM, so expect much slower inference.
- Qwen 2.5 72B Instruct72B · MMLU-Pro 71.1Q2_K · ~1.3 t/s
- Llama 3.3 70B Instruct70B · MMLU-Pro 68.9Q2_K · ~1.3 t/s
- DeepSeek R1 Distill Llama 70B70B · MMLU-Pro 70.0Q2_K · ~1.3 t/s
- Llama 3.1 70B Instruct70B · MMLU-Pro 66.4Q2_K · ~1.3 t/s
- Mixtral 8x7B Instruct v0.146.7B · MMLU-Pro 29.7Q4_K_M · ~1.4 t/s
- Command-R 35B35B · MMLU-Pro 33.0Q4_K_M · ~1.1 t/s
- Qwen 3.5 35B-A3B (MoE)35B · MMLU-Pro 85.3Q6_K · ~4.6 t/s
- Qwen 3.6 35B35B · MMLU-Pro 85.2Q6_K · ~1.2 t/s
- Ornith 1.5 35B-A3B (MoE)35B · MMLU-Pro N/AQ6_K · ~4.6 t/s
- Yi 1.5 34B Chat34.4B · MMLU-Pro 37.0Q6_K · ~1.2 t/s
- Qwen3 32B32.8B · MMLU-Pro 65.5Q6_K · ~1.4 t/s
- Qwen 2.5 32B Instruct32.5B · MMLU-Pro 69.0Q6_K · ~1.3 t/s
- Qwen 2.5 Coder 32B Instruct32.5B · MMLU-Pro 62.3Q6_K · ~1.3 t/s
- DeepSeek R1 Distill Qwen 32B32.5B · MMLU-Pro 65.0Q6_K · ~1.3 t/s
- Nemotron 3 Nano 30B32B · MMLU-Pro 78.3Q6_K · ~4.7 t/s
- Gemma 4 31B30.7B · MMLU-Pro 85.2Q6_K · ~1.5 t/s
- Qwen3 30B-A3B (MoE)30B · MMLU-Pro 61.5Q8_0 · ~3.2 t/s
- Nemotron 3.5 Lightning 30B-A3B30B · MMLU-Pro 81.6Q8_0 · ~3.4 t/s
- Muse Glimmer 30B27.8B · MMLU-Pro N/AQ8_0 · ~1.3 t/s
- Gemma 2 27B Instruct27.2B · MMLU-Pro 38.0Q8_0 · ~1.1 t/s
- Gemma 3 27B Instruct27B · MMLU-Pro 67.5Q8_0 · ~1.2 t/s
- Qwen 3.6 27B27B · MMLU-Pro 86.2Q8_0 · ~1.3 t/s
- UI-Mate 27B27B · MMLU-Pro ~86.2Q8_0 · ~1.3 t/s
- Qwen 3.8 27B27B · MMLU-Pro N/AQ8_0 · ~1.3 t/s
- Mistral Small 3.1 24B Instruct24B · MMLU-Pro 66.8Q8_0 · ~1.5 t/s
- Mistral Small 22B22.2B · MMLU-Pro 49.2Q8_0 · ~1.6 t/s
- GPT-OSS 20B21B · MMLU-Pro 67.9Q8_0 · ~3.5 t/s
Too large for this GPU (40)
- Mixtral 8x22B Instruct v0.1
- Llama 3.1 405B Instruct
- DeepSeek V3 671B
- DeepSeek R1 671B
- Llama 4 Scout 109B
- Llama 4 Maverick 400B
- Qwen3 235B-A22B (MoE)
- MiniMax M1 456B
- GPT-OSS 120B
- GLM-4.5 355B
- GLM-4.5 Air 106B
- GLM-4.6 355B
- GLM-4.6V 106B
- GLM-4.7 358B
- Qwen 3.5 122B-A10B (MoE)
- MiniMax M2.5 229B
- GLM-5 744B
- MiniMax M2.7 229B
- Nemotron 3 Super 120B
- Kimi K2.6
- GLM-5.1 754B
- DeepSeek V4 Pro 1.6T
- DeepSeek V4 Flash 284B
- Mistral Medium 3.5 128B
- GLM-5.2 753B
- Nemotron 3 Ultra 550B-A55B
- Step 3.5 Flash
- Step 3.7 Flash
- MiMo V2.5 Pro
- Kimi K2.5
- MiniMax M3
- Inkling
- Kimi K3
- DeepSeek V4 Flash 0731 284B
- Qwen3.8 2.4T-A95B
- Qwen3.8-Flash-Next
- DeepSeek V4 Pro 0813 1.6T
- Ornith 1.5 397B (MoE)
- GLM-5.3 753B
- GLM-5.3-Flash 320B
Compare NVIDIA RTX 3060 12GB with other GPUs
Continue reading
Frequently asked questions
- How much VRAM does the NVIDIA RTX 3060 12GB have?
- The NVIDIA RTX 3060 12GB has 12 GB of GDDR6 with 360 GB/s memory bandwidth.
- What is the NVIDIA RTX 3060 12GB best for?
- With 12 GB of VRAM, the NVIDIA RTX 3060 12GB comfortably handles 7B–8B models and can stretch to some 13B–14B models at aggressive quantization, a solid entry point into local LLM inference.
- What LLMs can the NVIDIA RTX 3060 12GB run locally?
- The NVIDIA RTX 3060 12GB can run 30 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Ornith 1.5 9B at Q8_0, Gemma 4 26B (MoE) at Q2_K, Qwen 3.5 9B at Q8_0.
- Can the NVIDIA RTX 3060 12GB run Gemma 4 31B?
- The NVIDIA RTX 3060 12GB can run Gemma 4 31B with CPU offload at Q6_K quantization, but inference will be slower than native VRAM execution.
- Can the NVIDIA RTX 3060 12GB run Qwen 3.6 27B?
- The NVIDIA RTX 3060 12GB can run Qwen 3.6 27B with CPU offload at Q8_0 quantization, but inference will be slower than native VRAM execution.
- Can the NVIDIA RTX 3060 12GB run Qwen3 8B?
- Yes. The NVIDIA RTX 3060 12GB runs Qwen3 8B natively in VRAM at Q8_0 quantization, achieving approximately 24.1 tokens per second.