Best LLMs for 16 GB VRAM GPUs (2026)

Why 16 GB Is the New Sweet Spot

A 16 GB card gives this site's calculator 15.2 GB to work with, because it holds back 5% of VRAM for the driver and display. Inside that budget, 2026 releases changed what fits. Nine-billion-parameter models such as Qwen 3.5 9B run at Q8_0 with room to spare, and their hybrid attention keeps the KV cache (the memory a model uses to remember the conversation) so small that a 128K-token context still fits at Q6_K. Dense 27B models are where 16 GB runs out. This guide keeps three groups strictly apart: models that fit at Q4_K_M or better, stretch picks that fit only below Q4, and larger mixture-of-experts (MoE) models that spill into system RAM, which are not a fit however fast they sometimes run.

Best All-Around: Qwen 3.5 9B and Gemma 4 12B

Qwen 3.5 9B (Alibaba, February 2026, Apache 2.0) is the pick for most people on a 16 GB card. At Q8_0 it needs 11.0 GB in total at 8K context, so you can run it near-lossless instead of squeezing into Q4_K_M (6.4 GB), and its hybrid attention keeps long contexts affordable, as the context section below shows. Alibaba reports 82.5 on MMLU-Pro and 81.7 on GPQA Diamond for it, and it accepts images as well as text. Thinking mode is on by default, so expect long reasoning traces unless you switch it off. The table shows the main contenders that fit at Q4_K_M or better, in the order this site's board ranks them by headline benchmark, with the older Qwen3 14B as a dense reference.

• Gemma 4 12B (Google, June 2026, Apache 2.0) is the choice if you need audio. It accepts text, images, audio clips of up to 30 seconds and short video, and Google reports 77.2 on MMLU-Pro and 78.8 on GPQA Diamond. Because 40 of its 48 layers attend over a 1,024-token window, Q4_K_M needs only 13.4 GB even at its full 256K window. Q8_0 fits at 8K (14.8 GB) but leaves just 0.4 GB spare, so Q6_K is the safer ceiling for long contexts.
• Phi-4 14B (December 2024) and Qwen3 14B (April 2025) still run well, but they score 70.4 and 61.03 on MMLU-Pro against 82.5 for Qwen 3.5 9B, and Phi-4 stops at a 16K native context, so they are now second choices.
• Ornith 1.5 9B ranks first on the board, but its score is a GPQA Diamond figure that Ornith AI reported itself and the model is tuned for agentic coding, so it has its own section below.

Best for Reasoning: GPT-OSS 20B

OpenAI's GPT-OSS 20B (August 2025, Apache 2.0) is a mixture-of-experts model with 21B total parameters but only 3.6B active per token, and a low, medium or high reasoning-effort setting that you choose in the system prompt (for example, "Reasoning: high"). OpenAI reports 71.5 on GPQA Diamond at high effort and says it runs within 16 GB; this site's math agrees, but only just. The Q4_K_M row totals 14.6 GB at 8K context, leaving 0.65 GB spare, and Q5_K_M no longer fits (17.0 GB). Because it reads only 3.6B parameters per token it decodes far faster than a dense model of the same size: Hardware Corner measured 82.4 tokens per second on an RTX 5060 Ti 16GB and 161.3 on an RTX 5080 at 16K context. The calculator estimates 116.8 and 198.3 for the same test, 23% to 43% above those measurements, because it has no term for decode cost that grows with context, so read its MoE speeds at long context as an upper bound; the llama.cpp scoreboard's shorter runs of this model sit much closer to the calculator. Its context ceiling is conservative too: the calculator stops near 30K tokens at Q4_K_M, while Hardware Corner ran the model at its full 128K context on the 5060 Ti.

Best for Coding at 16 GB: Ornith 1.5 9B

Ornith 1.5 9B (Ornith AI, August 2026, MIT) uses the same hybrid architecture as Qwen 3.5 9B, so it has the same footprint: 11.0 GB in total at Q8_0 and 8K context, and 13.1 GB at Q8_0 with a 64K context, which is the range coding agents actually want (Ollama's documentation recommends at least 64,000 tokens for agents and coding tools). Ornith AI's model card reports 70.6 on SWE-bench Verified and 46.2 on Terminal-Bench 2.1, against 53.2 and 21.3 for Qwen 3.5 9B in the same table. Those figures are self-reported by the developer, and we found no independent evaluation when this guide was updated, so read them as a promising signal rather than a ranking. For quick edits and chat, Qwen 3.5 9B remains the safer all-rounder.

• Ornith 1.5 35B-A3B: Ornith AI reports 79.0 on SWE-bench Verified. It is a mixture-of-experts model that does not fit 16 GB at Q4_K_M, so it needs the expert offload described below.
• Qwen 3.8 27B: Alibaba reports 73.0 on Terminal-Bench 2.1 and 61.7 on SWE-bench Pro. It fits only in 3-bit builds (see the stretch section). Qwen 3.6 27B has the identical footprint and a reported 77.2 on SWE-bench Verified.

How Much Context Fits on 16 GB

Weights are only half of the VRAM bill; the KV cache, which grows with every token of context, is the other half, and it depends on attention design rather than parameter count. A conventional model such as Llama 3.1 8B caches every token in all 32 layers, which comes to 17.2 GB at 131,072 tokens. Qwen 3.5 9B interleaves three Gated DeltaNet layers (linear attention with a fixed-size state) with every full-attention layer, so only 8 of its 32 layers cache anything: 4.3 GB at the same length. On a 16 GB card at Q4_K_M that is roughly 241K tokens against 64K, and it means Qwen 3.5 9B can hold the 128K tokens Alibaba recommends for thinking mode even at Q6_K (13.1 GB in total). Gemma 4 12B gets a similar result from sliding-window layers and reaches its whole 256K window at Q4_K_M (13.4 GB). The chart shows the longest context, in tokens, that stays inside 15.2 GB at Q4_K_M (Qwen 3.8 27B at Q3_K_M, its only fit).

Bigger MoE Models That Spill into System RAM

A mixture-of-experts model reads only a few experts per token, so it suffers far less from keeping most of its weights in slower system RAM than a dense model does. None of the models below fit in 15.2 GB at Q4_K_M, and this guide does not call them fits. The table shows how far over budget each one lands; the dense Qwen 3.8 27B is included to show what a spill looks like when the model is not an MoE.

• How to run them: llama.cpp can keep attention and shared layers on the GPU and park the expert tensors in system RAM with its -ncmoe option, which keeps the expert weights of the first N layers on the CPU, for example llama-server -m Qwen3.5-35B-A3B-Q4_K_M.gguf -c 16384 -ngl all -ncmoe 20 -fa on. Start with N around 20 for the 35B-A3B models, lower it until VRAM is nearly full, and raise it if you run out of memory. Plan on 32 GB of system RAM or more.
• Measured speed: Rost Glukhov tested the closely related Qwen 3.6 35B-A3B as a 17.7 GB IQ4_XS file on an RTX 4080 with 64 GB of RAM and llama.cpp's default layer offload, and saw about 95 tokens per second at 19K context, 92 at 32K and 86 at 64K. The calculator estimates 49.9 tokens per second for Qwen 3.5 35B-A3B at Q4_K_M on that card. It does model expert-only offload, but it assumes the spilled experts stream from system RAM at a conservative 25 GB/s, and Q4_K_M is a larger file than the IQ4_XS build tested, so treat that number as pessimistic here.
• Gemma 4 26B-A4B is the near miss, 2.4 GB over at Q4_K_M. Its IQ4_XS file (about 4.5 bits per weight, a little less faithful than Q4_K_M) is roughly 13.5 GB, and Glukhov measured it fully on the GPU at 121.7 tokens per second at 19K context on the same RTX 4080. Google reports 82.6 on MMLU-Pro for it.

Stretching to 27-31B Models

This is where 16 GB runs out. Dense 27B models carry 16.4 GB of weights at Q4_K_M, so Qwen 3.8 27B totals 19.0 GB at 8K context and spills 3.8 GB into system RAM, which is slow. On paper it fits only below Q4_K_M, and the Q3_K_M margin is razor thin: 15.15 GB against a 15.2 GB budget. Real files are also larger than the calculator's estimate. Unsloth's Qwen 3.6 27B Q3_K_M (the same architecture and footprint as Qwen 3.8 27B) is 13.59 GB against 12.99 GB estimated, which puts the real total near 15.8 GB, over budget. What does run fully on the GPU are 3-bit IQ3_XXS builds of about 11 to 12 GB. Treat all of this as a stretch pick, not a recommendation: this site recommends Q4_K_M or better, and everything in this section sits below it.

• Speed, independently measured: Glukhov ran Qwen 3.5 27B in a UD-IQ3_XXS file (11.5 GB) at about 45 tokens per second on an RTX 4080 at 19K to 32K context, dropping to 22.7 at 64K. A larger IQ4_XS file (15.0 GB) managed 20.5 tokens per second out of the box, and 38.0 after he hand-tuned layer offload at 18K context.
• Quality, independently measured: Quesma found Qwen 3.6 27B at Q4_K_M indistinguishable from BF16 in its tests. At 3 bits the results were mixed: Q3_K_M matched BF16 on AIME-120 (73.3% against 70.8%) but Q3_K_S fell to 54.2%, and Quesma concludes that 3-bit quality depends on the exact build. Unsloth's KL-divergence benchmark of the related Qwen 3.5 35B-A3B puts IQ3_XXS at about 0.05 mean KL divergence against about 0.019 for Q4_K_M, a small but measurable gap.
• Gemma 4 31B: Q4_K_M totals 22.6 GB and only Q2_K fits (14.8 GB), so skip it on 16 GB. Glukhov saw its 3-bit build fall from 29.2 to 8.1 tokens per second between 19K and 64K context.

GPU Comparison: Which 16 GB Card to Run LLMs On

Capacity decides what fits, and all twelve 16 GB discrete cards this site tracks have the same capacity, so they get identical fit results for every model above. Memory bandwidth decides speed, and it ranges from 224 GB/s on the Intel Arc Pro B50 to 960 GB/s on the RTX 5080; the bars show it for every card. Hardware Corner measured six of these cards in March 2026, and the table sets its Qwen3 14B figures beside the calculator's estimate for the same test, which lands within 8% of every measurement (never more than 8% low) and keeps the cards in the right order. By Hardware Corner's numbers the RTX 5080 is nearly three times as fast as the RTX 4060 Ti 16GB (64.0 against 22.4 tokens per second) and the RTX 5060 Ti 16GB is about 47% faster than the 4060 Ti.

• AMD and Intel cards have the same capacity, so the same models fit, but the calculator applies the same efficiency to every backend, so their estimates are probably optimistic. The exceptions are the RX 9070 XT and RX 7800 XT, which sit above the calculator's estimate on the llama.cpp project's public Vulkan scoreboard for a small dense model. ROCm is Linux-only on the RX 7900 GRE and RX 6800 XT (use the Vulkan backend on Windows), while AMD lists the RX 9070 XT and RX 7800 XT for the Windows HIP SDK as well, and Intel Arc support in Ollama is limited, per this site's GPU notes.
• The two newest AMD cards, the Radeon RX 9070 XT (640 GB/s) and the RX 7800 XT (624 GB/s), land in the table's upper half at 43.2 and 42.2 tok/s on the Qwen3 14B test, just behind the RTX 4070 Ti SUPER (672 GB/s, 45.1 tok/s).

Optimal Configurations by Use Case

Ollama's default context is 4K tokens on cards with under 24 GiB of VRAM, according to its documentation, so the first setting to change is the context length (set OLLAMA_CONTEXT_LENGTH before starting the server). The second is to run ollama ps and confirm nothing is sitting partly on the CPU. The commands below cover each use case, and the table after them gives the calculator's total at the stated context.

# Raise Ollama's context length first (the default is 4K under 24 GiB of VRAM)
OLLAMA_CONTEXT_LENGTH=32768 ollama serve

# Daily assistant
ollama run qwen3.5:9b-q8_0

# Long documents (restart the server with OLLAMA_CONTEXT_LENGTH=131072)
ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q6_K

# Coding agent (restart the server with OLLAMA_CONTEXT_LENGTH=65536)
ollama run hf.co/ornith-ai/Ornith-1.5-9B-GGUF:Q8_0

# Audio and video input (restart the server with OLLAMA_CONTEXT_LENGTH=32768)
ollama run hf.co/unsloth/gemma-4-12B-it-GGUF:Q6_K

# Reasoning (set the effort in the system prompt: Reasoning: low, medium or high)
ollama run gpt-oss:20b

# Stretch: 27B in a 3-bit build
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-IQ3_XXS

# Not a fit: Qwen 3.5 35B-A3B with experts in system RAM (32 GB or more); tune -ncmoe
llama-server -m Qwen3.5-35B-A3B-Q4_K_M.gguf -c 16384 -ngl all -ncmoe 20 -fa on