Best LLMs for 12 GB VRAM (2026): RTX 3060, 4070 & 5070

The 12 GB Tier: Where Local LLMs Get Interesting

A 12 GB card is where local LLMs stop being a toy. This site holds back 5% of the spec-sheet VRAM for the driver and runtime, so a 12 GB GPU offers a model 11.4 GB, and that budget has to cover three things: the weights (the model file), the KV cache (working memory that grows with every token of context) and roughly 12% overhead. Quantization shrinks the weights: Q4_K_M, the common default, stores them in roughly 4.9 bits each, while Q8_0 is near-lossless at about 8.5. Every figure on this page is that full total at an 8K-token context with a 16-bit KV cache, unless a section says otherwise. The short version: 9B to 12B dense models fit with room to spare, 14B models sit right at the wall, and 30B-class mixture-of-experts (MoE) models run only with part of the model pushed out to system RAM.

Best Dense Models for 12 GB: Qwen 3.5 9B, Gemma 4 12B and Ornith 1.5 9B

A dense model uses all of its parameters for every token, so its whole file has to sit in VRAM to run at full speed. Ranked the way this site's calculator orders models (verdict first, then headline benchmark score), the current dense models that fit a 12 GB card in full at Q4_K_M or better lead with Ornith 1.5 9B and Qwen 3.5 9B, then Gemma 4 12B; the ternary Bonsai 27B build also ranks high and gets its own treatment at the end of this page. The two 9B models share one architecture, a hybrid stack in which only 8 of 32 layers keep a KV cache, so they stay cheap at long context; Alibaba's model card reports 82.5 on MMLU-Pro for Qwen 3.5 9B, with thinking mode on by default. Gemma 4 12B, from Google under Apache 2.0, reports 77.2 on MMLU-Pro, takes text, image and audio input, and keeps its cache small a different way: 40 of its 48 layers look back only 1,024 tokens. It tops out at Q5_K_M here, because its Q6_K build totals 11.56 GB, 0.16 GB over the line. The table gives each model's highest quant that fits at 8K next to its Q4_K_M total, with the older 14B class for contrast.

How Much Context Fits: The KV Cache Ceiling

Weights are a fixed cost; the KV cache is the part that grows. At 16-bit precision every token of context adds a fixed number of bytes per caching layer, so a model that caches every layer runs out of room fast, while hybrid and sliding-window designs barely notice. The chart shows how many tokens of context each model keeps at Q4_K_M before a 12 GB card hits its 11.4 GB ceiling, using the calculator's own KV math. Qwen 3.5 9B (and Ornith 1.5 9B, which shares its architecture) reaches 143K tokens and Gemma 4 12B reaches 155K, while the older full-attention models stop far sooner: Mistral Nemo 12B at 17K, Phi-4 at 8K (half of its 16K native window) and Qwen3 14B at 7K.

The 14B Question: Phi-4, Qwen3 14B and the 11.4 GB Wall

Older advice says a 12 GB card 'unlocks' 14B models at Q4. By this calculator's math they sit right at the wall instead. Phi-4's Q4_K_M weights are 8.53 GB; add its 1.68 GB KV cache at 8K context (10 KV heads make it heavier than most 14B models) and the total is 11.43 GB, 0.03 GB over the 11.4 GB budget. Qwen3 14B totals 11.6 GB. Both run fine at a shorter context, but neither holds a full 8K conversation without spilling. Real files drift from the generic ladder too: the bartowski Phi-4 Q4_K_M GGUF is 9.05 GB on Hugging Face, about 0.5 GB larger than the calculator's 8.53 GB, so treat Phi-4 as a short-context model on this tier. Microsoft's model card also describes it as aimed at English, which narrows its appeal further. Gemma 4 12B or a 9B model is the safer pick; if you still want a 14B, you have three levers.

• Cut the context to 4K: Phi-4 at Q4_K_M then totals 10.49 GB and fits.
• Quantize the KV cache to q8_0, which roughly halves it. The totals above assume a 16-bit cache.
• On an RTX 5070 only, use NVFP4, the 4-bit format native to Blackwell cards: Phi-4 then totals 9.72 GB at an estimated 58.6 tok/s, provided your runtime and model file support it.

The Q8 7-9B Option: Quality Over Quantity

With 12 GB you can run a 9B model at Q8_0, close to unquantized quality, instead of the usual Q4_K_M. Qwen 3.5 9B at Q8_0 totals 11.02 GB at 8K context, a real fit with only 0.38 GB to spare, and the table shows the price: the context ceiling falls from 143K tokens at Q4_K_M to 19K at Q8_0. Q6_K is the balance point for most people: 8.58 GB total, room for about 85K tokens, and about 26% faster than Q8_0 on the same card, because decode speed scales with the bytes read per token. Q8_0 earns its cost on work where small precision losses show up, such as code generation, structured extraction and math; for ordinary chat the gap between Q6_K and Q8_0 is small. Ornith 1.5 9B has the identical shape, so the same numbers apply to it.

MoE Models on 12 GB: Big Capability, but Only With Partial Offload

Mixture-of-experts models hold many specialist sub-networks but use only a few per token, so they decode fast for their size, yet the whole file still has to live somewhere. On a 12 GB card the honest verdict for every 20B-plus MoE model at Q4_K_M is offload, not fit: none gets under the 11.4 GB line, and the table below shows how far past it each one reaches, in the calculator's ranking order. What makes them usable anyway is llama.cpp's expert offload, the n-cpu-moe setting (ncmoe for short), which keeps the expert tensors of the first N layers in system RAM while attention and the shared weights stay on the GPU. The command shows its shape.

# Experts of the first 24 layers stay in system RAM
llama-server -m model.gguf -ngl 99 -ncmoe 24 -c 8192 -fa on
# Out of memory: raise -ncmoe. VRAM to spare: lower it.

• System RAM sets the speed once experts spill, and the spilled part has to fit: plan for 32 GB (this calculator's baseline) rather than 16 GB.
• Tune N by trial: raise ncmoe until the model loads with a little VRAM to spare, then lower it one or two steps at a time. Each step moves one layer's experts onto the GPU.
• GPT-OSS 20B is the gentlest case at 3.15 GB past the line. OpenAI's model card says it runs within 16 GB of memory, so a 12 GB card needs a little offload.

GPU Comparison: RTX 3060 vs Arc B580 vs RTX 4070 vs RTX 5070

Capacity is identical across these cards, so the choice is speed, software and price. Decode speed follows memory bandwidth: the chart is the calculator's estimate, in tokens per second, for Qwen 3.5 9B at Q4_K_M, from 49.8 on an RTX 3060 to 85.9 on an RTX 5070. Independent llama.cpp results point the same way for NVIDIA, with one caveat for Intel.

• On the CUDA scoreboard (llama.cpp discussion 15013), Llama 2 7B Q4_0 token generation runs at about 77 tok/s on an RTX 3060 and 128 on an RTX 5070.
• On the Vulkan scoreboard (discussion 10879) the same test gives about 92 for an RTX 4070, 81 for an RTX 3060 and 72 for an Arc B580. The B580 has 27% more bandwidth than the 3060 on paper but trails it by roughly 8 to 11% there, so the calculator's bandwidth-driven estimate (61.5 against 49.8 tok/s for Qwen 3.5 9B at Q4_K_M) flatters Arc.
• Arc runs llama.cpp through its Vulkan backend, which reports generally find easier than SYCL. CUDA-only tools such as ExLlamaV2 and TensorRT-LLM are off the table.
• The RTX 3060 12GB remains the VRAM-per-dollar floor, and the 4070 family adds speed at the same capacity. Price trackers have shown some of these cards above MSRP this year, so check current listings, and price a used 24 GB card before paying new-card money for 12 GB.

Stretching to 27-32B Models

Dense 27B to 32B models are the stretch. Qwen 3.6 27B at Q4_K_M totals 19.02 GB at 8K, 7.62 GB past the line, and the calculator estimates 3.8 tok/s on an RTX 3060 (4.1 on an RTX 5070), too slow to chat with. Lower quants do not rescue it: Q3_K_M still totals 15.15 GB, and even Q2_K totals 12.12 GB, still over the line at a quality cost not worth paying. Gemma 4 31B is heavier at 22.63 GB at Q4_K_M. Two routes do work.

• The MoE route above: a 35B-A3B model trades speed for strong benchmark scores.
• A ternary build such as PrismML's Bonsai 27B, derived from Qwen3.6-27B with three-valued weights: 7.2 GB of weights, 8.67 GB total at 8K and an estimated 35.4 tok/s on an RTX 3060. PrismML reports that it keeps 94.6% of the base model's score across its own 15 benchmarks, a vendor figure not yet independently verified.