Can I run Mistral Large 2 locally?

No — API-only via Mistral's platform. For a local large model, use Llama 3.3 70B or Qwen 2.5 72B.

Why does Mixtral 8x7B need 28 GB at Q4?

All 46.7B expert parameters must be loaded even though 12.9B are active per token. MoE saves compute, not memory.

Mistral 7B vs Llama 3.1 8B?

Very close. Mistral 7B uses less VRAM (~4.4 vs ~5 GB) and has sliding window for long docs. Llama 3.1 8B has better benchmarks and ecosystem. Either is excellent.

Best Mistral for 16 GB?

Small 3.1 24B at Q4 (~14 GB) — fits with room for context, adds vision. NeMo 12B at Q8 for near-lossless text-only.

Mistral & Mixtral Family: Complete VRAM Reference

CanItRunJuly 1, 20269 min readVRAM Guides

Mistral AI: The French Challenger

Mistral AI's portfolio spans dense (7B-123B) and MoE (46B-141B) models. Key distinction: Mistral Large 2 (123B) is API-only, not downloadable. Open-weight models: Mistral 7B, NeMo 12B, Small 22B, Small 3.1 24B, Mixtral 8x7B, Mixtral 8x22B. Recent 2026 additions (Medium 3.5 128B, Small 4 119B) have minimal coverage elsewhere.

Dense Models

Mistral 7B Q4 (~4.4 GB) — lightweight entry. NeMo 12B Q4 (~7.5 GB) — designed for consumer GPUs, excellent performance-per-VRAM. Small 22B Q4 (~13 GB). Small 3.1 24B Q4 (~14 GB, 2026) — fits 16 GB GPUs, adds vision. The 24B on 16 GB is the Mistral sweet spot.

Mixtral MoE: Just Past Consumer Reach

Mixtral 8x7B (46.7B total, 12.9B active) at Q4 needs ~28 GB — just past single RTX 4090. Partial offloading works. Mixtral 8x22B (141B total, 39B active) at Q4 needs ~80 GB — workstation territory. UNIQUE: no competitor site has VRAM-by-quantization data for 8x22B.

What Can You Actually Download?

Apache 2.0: Mistral 7B, NeMo 12B, Small 22B, Mixtral 8x7B, Mixtral 8x22B. Research License (non-commercial): Small 3.1 24B. API-only (no weights): Mistral Large 2. Always verify license before planning deployments.

Which Mistral for Your GPU?

8 GB: Mistral 7B Q4 (~4.4 GB) or Q5. 12-16 GB: NeMo 12B Q4 (~7.5 GB) or Small 3.1 24B Q4 (~14 GB, needs 16 GB). 24 GB: Mixtral 8x7B Q4 (~28 GB, partial offload). 48 GB+: Mixtral 8x22B Q4 (~80 GB). NeMo 12B is the practical pick for most users.

Frequently asked questions

Can I run Mistral Large 2 locally?: No — API-only via Mistral's platform. For a local large model, use Llama 3.3 70B or Qwen 2.5 72B.
Why does Mixtral 8x7B need 28 GB at Q4?: All 46.7B expert parameters must be loaded even though 12.9B are active per token. MoE saves compute, not memory.
Mistral 7B vs Llama 3.1 8B?: Very close. Mistral 7B uses less VRAM (~4.4 vs ~5 GB) and has sliding window for long docs. Llama 3.1 8B has better benchmarks and ecosystem. Either is excellent.
Best Mistral for 16 GB?: Small 3.1 24B at Q4 (~14 GB) — fits with room for context, adds vision. NeMo 12B at Q8 for near-lossless text-only.