Best LLMs You Can Run on 8 GB VRAM (2026)

What 8 GB of VRAM Actually Gets You

Eight gigabytes of VRAM puts you in the domain of models with roughly 1 to 8 billion parameters at practical quantization levels. This is not a limitation to dismiss: 7B-class models in 2026 are dramatically more capable than the 175B models of just a few years ago, thanks to improvements in training data quality, architecture, and alignment. At Q4_K_M quantization, a 7B model consumes approximately 4.5 GB for weights, leaving about 3 GB for the KV cache, runtime overhead, and whatever else your GPU is doing. That is enough headroom for roughly 4096-8192 tokens of context, depending on the model's architecture. You will not be running 14B or larger models on 8 GB in any practical sense. Even at the lowest usable quantization (Q2_K), a 14B model needs about 6 GB for weights alone, leaving almost no room for inference. Accept this constraint and focus on finding the best 7B-class model for your specific use case rather than trying to squeeze a bigger model into too little memory. If you're shopping rather than working with a card you already own, it's worth seeing how close a real step up in VRAM is in bandwidth terms: the RTX 3060 12GB, RX 7900 GRE 16GB, and Arc A770 16GB below all have more capacity than the RTX 4060's 8 GB:

Best Overall: Qwen 3.5 9B

As of mid-2026, the community default for 8 GB has shifted to Qwen 3.5 9B. Its hybrid Gated DeltaNet / Gated Attention stack (only 8 of its 32 layers keep a growing KV cache) means the weights are slightly bigger than a plain 8B model (about 5.5 GB at Q4_K_M), but the KV cache barely grows with context. The practical effect: it comfortably beats Llama 3.1 8B on reasoning and coding benchmarks while also leaving more headroom for long context on the same 8 GB card, not less. Llama 3.1 8B is still worth knowing: it remains the safest, most universally-compatible default: every inference engine, GUI frontend, and API wrapper supports it out of the box, and its ecosystem of fine-tunes and guides is unmatched. If you want the strongest model this tier can offer, install Qwen 3.5 9B first; if you hit a compatibility wall with some tool, Llama 3.1 8B is the reliable fallback. The gap becomes obvious once you push context past a few thousand tokens:

# Install and run Qwen 3.5 9B with Ollama
ollama run qwen3.5:9b

# The safe, universally-compatible fallback
ollama run llama3.1:8b

# Check VRAM usage after loading
nvidia-smi

Best for Coding: Qwen 3.5 9B

The dedicated small coder models that used to own this slot (Qwen 2.5 Coder 7B chief among them) have beenovertaken by current-generation generalists. Qwen 3.5 9B, the same model recommended above for overall use, now beats the old Coder 7B on real coding benchmarks while also handling chat, reasoning, and instruction-following the dedicated coder never could. This mirrors a broader 2026 trend across the Qwen line: each new generalist release lands ahead of the previous generation's dedicated coder variant, so there is less reason than there used to be to install a separate, narrower model just for code. At Q4_K_M it uses the same ~5.5 GB as the general-use configuration above. If you specifically need a lighter, more universally-compatible fallback, Qwen3 8B (Q4_K_M, ~4.9 GB) is a solid dense alternative with wide tool support. For pure coding work on a budget GPU, install Qwen 3.5 9B first: you no longer need a separate coder model on this tier.

# Run Qwen 3.5 9B for coding (same model as general use)
ollama run qwen3.5:9b

# Lighter dense fallback
ollama run qwen3:8b

# Use with continue.dev or other IDE integrations
# Set the model to qwen3.5:9b in your IDE extension settings

Best for Reasoning: Phi-4-mini and Gemma 4 E4B

For users who want reasoning capability on 8 GB VRAM, Phi-4-mini remains a strong pick at 3.8B: it uses about 2.3 GB at Q4_K_M for weights, and its grouped-query attention (8 KV heads instead of Phi-3.5 Mini's uncompressed 32) cuts its KV cache to roughly a third of Phi-3.5 Mini's at any given context length: at 32K tokens that's about 4.3 GB of KV cache versus 12.9 GB for the old model, the difference between comfortably fitting an 8 GB card and blowing well past it. It also adds function calling and a larger vocabulary. It will not match a 70B model on complex multi-step reasoning, but for straightforward math, logic puzzles, and structured analysis, it is surprisingly good. Gemma 4 E4B, released by Google in June 2026, is the newer option worth trying alongside it: at the same 4B-class size and about 2.4 GB at Q4_K_M, it scores 69.4 on MMLU-Pro, a substantial jump over the Gemma 3 4B it replaces (43.6), and inherits Gemma 4's encoder-free multimodal design, so it can also read images at this size. Both models are small enough that you can comfortably run them at Q8_0 on an 8 GB card, which is rare for this VRAM tier and means you get near-lossless quality.

# Run Phi-4-mini for reasoning tasks
ollama run phi4-mini:3.8b

# Run Gemma 4 E4B as an alternative
ollama run gemma4:e4b

Quantization Trade-offs at 8 GB

At the 8 GB tier, your quantization choice meaningfully affects both quality and usability. Q4_K_M is the default recommendation for 7B models because it preserves the vast majority of model quality while leaving enough VRAM for practical context lengths. Going to Q3_K_M saves roughly 1 GB but introduces noticeable degradation: you will see more grammatical errors, less coherent long-form output, and weaker performance on knowledge-intensive tasks. Q2_K is generally not recommended for 7B models because the quality loss is substantial. Going the other direction, Q5_K_M adds roughly 0.8 GB over Q4_K_M and gives a small but real quality improvement, and Q8_0 roughly doubles the Q4_K_M footprint for near-lossless quality; the chart below shows how each level scales relative to full FP16. If you have exactly 8 GB and want the best quality, use Q4_K_M on a 7B model. If you have 10-12 GB, step up to Q5_K_M or even Q8_0. One important caveat: not all 7B models are created equal. Some models have larger embedding tables or different architectures that shift these numbers by a few hundred megabytes. Always check the actual file size of the quantized GGUF before assuming it will fit.

Context Length Limitations on 8 GB

Context length is where 8 GB VRAM really feels constraining. After loading a 7B model at Q4_K_M (approximately 4.5 GB), you have roughly 3 GB left for the KV cache and runtime overhead. The runtime overhead consumes about 500 MB, leaving 2.5 GB for the KV cache. For a model like Llama 3.1 8B with grouped-query attention, this translates to roughly 8K-12K tokens of context at FP16 KV cache precision. If you enable KV cache quantization (supported in recent versions of llama.cpp and Ollama), you can stretch this to 16K-24K tokens. The practical impact: you can have a moderately long conversation, process a short document, or work with a few files of code. You cannot process a full novel, a long technical specification, or maintain context across an extended multi-turn session without the model forgetting earlier parts. If you frequently need long context, upgrading to a 12 or 16 GB card is a better investment than trying to optimize around the 8 GB limit.

Practical Setup Tips for 8 GB Cards

To get the most out of an 8 GB GPU for local LLMs, start with a few practical optimizations. First, close your browser or disable GPU acceleration in browser settings. Chrome and Firefox can consume 500 MB to 2 GB of VRAM for hardware-accelerated rendering, eating directly into your LLM headroom. Second, disable any overlays (Discord, Steam, GeForce Experience) that consume VRAM. Third, set your desktop to a single monitor if you are running multi-display. Fourth, on Linux, consider using a lightweight desktop environment or even a TTY session for inference to minimize background VRAM usage. Ollama is the easiest way to get started: it handles model downloading, quantization selection, and GPU management automatically. For more control, llama.cpp lets you specify exact VRAM budgets with the -ngl (layers to GPU) flag. If a model does not quite fit, try reducing -ngl by 5-10 layers instead of switching to a smaller model. Partial offloading to CPU is slower, but the layers that remain on GPU still benefit from GPU acceleration. On AMD cards, make sure ROCm is properly configured (Linux) or use the Vulkan backend (Windows) for GPU acceleration.

# Check current VRAM usage before running a model
nvidia-smi

# Kill GPU-hungry background processes
sudo fuser -v /dev/nvidia*

# Run with explicit layer control in llama.cpp
./llama-cli -m model-Q4_K_M.gguf -ngl 33 -c 4096

Models to Skip on 8 GB VRAM

Not every model marketed as small actually fits on 8 GB. Models with 12-14B parameters, even at Q3_K_M, typically need 7-9 GB for weights alone, leaving no room for inference. Mixture-of-experts models like Mixtral 8x7B have 46.7B total parameters (12.9B active), and their weight files are sized by total parameters, not active ones, so they need 24+ GB even at Q4_K_M. Any model with a 128K default context length will try to allocate a large KV cache on startup, which can cause out-of-memory errors before inference even begins. Reduce the context length explicitly. Also be cautious with vision-language models at this tier: the image encoder adds substantial overhead beyond the text model weights. If you want multimodal capability on 8 GB, Gemma 4 E4B's encoder-free unified design is one of the few that fits, though you will have very limited context for the text portion during image analysis.