CanItRun Logocanitrun.

Qwen 2.5 Coder 32B Instruct

Qwen 2.5 Coder 32B Instruct needs roughly 24.6 GB VRAM at Q4_K_M quantization (75.2 GB at FP16). 70 GPUs we track can run it fully in VRAM at 8k context.

70 GPUs run this natively · 31 with CPU offload

Alibaba32.5B params128k contextApache 2.0Commercial use ok
Q4_K_M total
24.6 GB
at 8k context
Smallest GPU
32 GB
NVIDIA RTX 5090, at Q4_K_M
KV cache, full context
34.4 GB
64 of 64 layers cache
Inputs
Text
Apache 2.0, released November 2024
Aider (whole-edit)
73.7%
vs GPT-4o's 74.4%, same table
Context default
32,768 tokens
131,072 needs manual YaRN setup

Qwen 2.5 Coder 32B Instruct is a 32.5B parameter dense model developed by Alibaba. Released November 12, 2024 as a dense 32.5B-parameter fine-tune of the Qwen2.5 architecture: grouped-query attention (GQA) with 8 KV heads over 64 layers, trained on more than 5.5 trillion tokens weighted heavily toward code. Licensed Apache 2.0 with no usage restrictions; the only sibling size under a different license is the 3B, which ships non-commercial. Context ships at 32,768 tokens by default; the model supports up to 131,072 via YaRN, but that needs a rope_scaling block manually added to config.json, and Qwen's own docs recommend enabling it only when long context is actually needed, since static YaRN trades away a little short-context quality.

To run Qwen 2.5 Coder 32B Instruct locally: Q4_K_M's real total at this site's 8k-token baseline is 24.6GB, against a 24GB card's usable ceiling of roughly 22.8GB, so an RTX 4090 or RTX 3090 needs either a short context window (roughly 2,000 tokens or less) or a small CPU offload to run it. Q3_K_M (19.9GB) is the quant that actually fits a 24GB card cleanly with room to spare; a 32GB+ card (RTX 5090, RTX 5000 Ada) is what runs Q4_K_M comfortably at a full working context.

Qwen's own technical report puts HumanEval at 92.7 and MBPP at 90.2, and Aider's harder "whole-file edit" pass rate at 73.7, voluntarily compared by Qwen to GPT-4o-2024-08-06's 74.4 on the same table, a genuinely close result. MMLU-Pro is 62.3, respectable general reasoning for a model this narrowly specialized on code. These are vendor-measured figures from Qwen's own arXiv paper, not an independent leaderboard.

VRAM at each quantization

Figures below assume 8k context; KV cache grows linearly as context length increases.

QuantWeightsKV cacheTotal
FP32130.0 GB2.15 GB148.0 GB
BF1665.0 GB2.15 GB75.2 GB
FP1665.0 GB2.15 GB75.2 GB
Q8_034.5 GB2.15 GB41.1 GB
Q6_K26.7 GB2.15 GB32.3 GB
Q5_K_M23.1 GB2.15 GB28.3 GB
Q4_K_Mrec19.8 GB2.15 GB24.6 GB
Q3_K_M15.6 GB2.15 GB19.9 GB
Q2_K12.4 GB2.15 GB16.3 GB
NVFP4cuda16.3 GB2.15 GB20.6 GB

Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.

Quantization × context: what the YaRN-extended window actually costs

The table above holds context at 8k. Add context as a second axis and the real shape of this model's context story becomes visible: from the 32,768-token default it ships with, up to the full 131,072-token window that only exists once YaRN is manually enabled in config.json. Pick the row you can download, then the column that matches how you've actually configured it.

Total VRAM needed by quantization and context length, with the GPU class each figure requires.
Quant8k ctx32k ctx128k ctx
Q8_041.1 GB48 GB card48.3 GB80 GB card77.2 GB80 GB card
Q6_K32.3 GB48 GB card39.5 GB48 GB card68.4 GB80 GB card
Q5_K_M28.3 GB32 GB card35.5 GB48 GB card64.4 GB80 GB card
Q4_K_Mrec24.6 GB32 GB card31.8 GB32 GB card60.6 GB80 GB card
Q3_K_M19.9 GB24 GB card27.1 GB32 GB card56.0 GB80 GB card
Each cell shows the smallest card that holds it:Consumer GPUWorkstation GPUDatacenter GPU

Benchmarks

MMLU-Pro
62.3
GPQA Diamond
41.8
IFEval
79.9
MATH
76.4
LiveCodeBench
31.4

LiveCodeBench: a late-2024 coding specialist against a newer field

LiveCodeBench is deliberately a moving target: it rotates in new, previously-unseen problems over time so models can't have trained on the test set. Qwen's reported score here, 31.4, comes from a contamination-controlled slice of problems from July-November 2024, chosen specifically because they postdate this model's training cutoff, not the full accumulated problem set some other entries here may report against. Plotted by size against every other tracked model that reports this benchmark, it lands toward the lower end: most of its neighbors are newer models evaluated on a LiveCodeBench that has grown harder as frontier models caught up to earlier versions of it. Read this chart as "a coding specialist from late 2024," not a same-day comparison against newer generalist flagships.

25609510B100B1T10TTotal parameters (log scale)Qwen 2.5 Coder 32B InstructDeepSeek V3 671BDeepSeek R1 671BLlama 4 Maverick 400BGLM-4.6 355BGemma 4 31BGemma 4 26B (MoE)Qwen 3.5 122B-A10B (MoE)Qwen 3.6 27BKimi K2.6Step 3.5 FlashKimi K2.5Qwen 3.8 27BQwen3.8-Flash-Next
LiveCodeBench, as reported by each model's own publisher. Scaffolding and harness differ between vendors, so read the gaps as approximate rather than a ranking.

GPUs that run Qwen 2.5 Coder 32B Instruct natively (70)

Show 65 more
Plus 31 GPUs that run it with CPU offload (slower)

Notes

Best open-weight coding model at this size.

Hugging Face ↗Ollama ↗Released 2024-11-12

Compare Qwen 2.5 Coder 32B Instruct with other models

How to run Qwen 2.5 Coder 32B Instruct locally

816244880160320

Q4_K_M needs 24.6 GB: needs a workstation or datacenter GPU (48–80 GB).

Ollama

ollama run qwen2.5-coder:32b

llama.cpp

./llama-cli -m qwen2.5-coder-32b-instruct.Q4_K_M.gguf -c 8192 -ngl 99

LM Studio: Search for 'Qwen 2.5 Coder 32B' in LM Studio. Q4_K_M needs about 24.6 GB at a full 8k context, just over what a 24 GB card (RTX 4090, RTX 3090) has usable, so expect a small CPU offload or a shorter context window unless you step up to 32 GB. Q3_K_M (19.9 GB) is the quant that actually fits a 24 GB card cleanly.

Why this quantization? At 32.5B dense parameters, Q4_K_M keeps enough precision for accurate code synthesis while staying close to a single high-end consumer GPU's ceiling, though its real total at 8k context (24.6 GB) runs just over a 24 GB card's usable ~22.8 GB, so a 24 GB card wants a shorter context window or a small offload, and a 32 GB card (RTX 5090, RTX 5000 Ada) is what runs it fully in VRAM with real headroom. This model was specifically trained for code generation, so its performance is disproportionately strong for its size relative to general-purpose models at the same quant.

Who is Qwen 2.5 Coder 32B Instruct for?

Software developers who want the best local coding assistant that fits on a single high-end consumer GPU. Ideal for programmers who want to pair it with VS Code or a terminal-based workflow, and who value the Apache 2.0 license for integrating AI-assisted coding into commercial projects.

Best for

  • Code generation, completion, and refactoring across popular languages
  • Explaining and documenting existing codebases
  • Unit test generation and bug detection
  • Technical writing and API documentation
  • Pair programming with local privacy -- your code never leaves your machine

Not ideal for

  • General-purpose chat or creative writing -- this model is specialized for code
  • Users with less than 20 GB of VRAM -- consider Qwen 2.5 7B Coder or DeepSeek R1 Distill 8B
  • Tasks requiring strong multilingual natural-language understanding outside of code

So should you run it?

On a 32GB+ card (RTX 5090, RTX 5000 Ada, or similar) this is a straightforward pick for local coding work: Q4_K_M runs fully in VRAM with real context headroom, and Aider and HumanEval scores that Qwen puts within a point of GPT-4o are hard to match at this parameter count. On a 24GB card (RTX 4090, RTX 3090) the honest recommendation is Q3_K_M, not Q4_K_M: Q4's real total edges just past a 24GB card's usable ceiling once KV cache and activation overhead are counted, so either drop one quant tier or keep the context window short. Whichever GPU you have, remember the 131,072-token context is opt-in: the model defaults to 32,768, and reaching the full window means manually adding a YaRN config block, not just requesting a bigger number from your inference server.

Continue reading

vram-guides10 min
Qwen Family Guide: 2.5 to 3.6
hardware10 min
Best GPUs for Coding Agents in 2026

Frequently asked questions

What are the VRAM requirements for Qwen 2.5 Coder 32B Instruct?
Qwen 2.5 Coder 32B Instruct requires approximately 24.6 GB of VRAM at Q4_K_M quantization, 41.1 GB at Q8, and 75.2 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Qwen 2.5 Coder 32B Instruct have?
Qwen 2.5 Coder 32B Instruct has 32.5 billion parameters.
Is Qwen 2.5 Coder 32B Instruct good for coding?
Yes. Qwen 2.5 Coder 32B Instruct scores 31.4 on LiveCodeBench, demonstrating strong code generation and completion capabilities.
Can Qwen 2.5 Coder 32B Instruct run on a 16 GB GPU?
No. At Q4_K_M, Qwen 2.5 Coder 32B Instruct needs 24.6 GB of VRAM, more than 16 GB. You will need a 32 GB GPU like the RTX 5090.
Can Qwen 2.5 Coder 32B Instruct run on a 24 GB GPU?
No. Even at Q4_K_M, Qwen 2.5 Coder 32B Instruct needs 24.6 GB. Consider a 32 GB card like the RTX 5090.
What is the smallest quantization for Qwen 2.5 Coder 32B Instruct that fits in 24 GB of VRAM?
At NVFP4, Qwen 2.5 Coder 32B Instruct needs 20.6 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Qwen 2.5 Coder 32B Instruct locally?
A 32 GB GPU is the minimum. At Q4_K_M, Qwen 2.5 Coder 32B Instruct needs 24.6 GB VRAM. Good option: RTX 5090 (32 GB).
Does Qwen2.5-Coder-32B really support 128K (131,072-token) context?
Yes, but not out of the box. The model's default config.json caps out at 32,768 tokens; YaRN scaling to the full 131,072-token window has to be added manually as a rope_scaling block, which Qwen's own documentation recommends enabling only when you actually need long context, since static YaRN trades away a little quality on shorter prompts. Most default deployments (a fresh Ollama pull, an unmodified vLLM config) run at the 32,768-token default unless you make that change yourself.
Does Qwen2.5-Coder-32B fit on a 24GB GPU like the RTX 4090?
At Q3_K_M, yes, cleanly: about 19.9GB at this site's 8k-token baseline. Q4_K_M, the quant this model recommends by default, does not: its real total is about 24.6GB, just over a 24GB card's usable ~22.8GB, so an RTX 4090 or RTX 3090 needs either a short context window (roughly 2,000 tokens or less) or a small CPU offload to run Q4_K_M. A 32GB card runs Q4_K_M fully in VRAM with real headroom.
How does Qwen2.5-Coder-32B compare to GPT-4o for coding?
Close, on Qwen's own numbers: 92.7 vs GPT-4o-2024-08-06's 92.1 on HumanEval, and 73.7 vs 74.4 on Aider's harder whole-file-edit benchmark, a comparison Qwen highlights directly in its own technical report, not an independent leaderboard result. Where the two diverge more is on newer, harder suites: GPT-4o edges ahead on LiveCodeBench and BigCodeBench in the same report. Treat "comparable to GPT-4o" as true for the specific benchmarks Qwen chose to lead with, not a blanket claim.