CanItRun Logocanitrun.

Command-R 35B

Command-R 35B needs roughly 35.9 GB VRAM at Q4_K_M quantization (90.4 GB at FP16). 47 GPUs we track can run it fully in VRAM at 8k context.

47 GPUs run this natively · 36 with CPU offload

Cohere35B params125k contextCC-BY-NC 4.0Non-commercial only

Command-R 35B is a 35B parameter dense model developed by Cohere. August 2024 RAG and tool-use specialist from Cohere. 128K context with full attention (no GQA).

To run Command-R 35B locally: Q4_K_M ~20-22GB — fits on 24GB GPU but full attention means heavy KV cache at long context. 32GB+ recommended for RAG workloads.

Industry-leading retrieval-augmented generation. Strong multilingual support across 10 languages.

VRAM at each quantization

Calculated at 8k context. Since KV cache scales linearly with context, longer sessions need more VRAM than shown here.

QuantWeightsKV cacheTotal
FP32140.0 GB10.74 GB168.8 GB
BF1670.0 GB10.74 GB90.4 GB
FP1670.0 GB10.74 GB90.4 GB
Q8_037.2 GB10.74 GB53.7 GB
Q6_K28.7 GB10.74 GB44.2 GB
Q5_K_M24.9 GB10.74 GB39.9 GB
Q4_K_Mrec21.3 GB10.74 GB35.9 GB
Q3_K_M16.8 GB10.74 GB30.9 GB
Q2_K13.3 GB10.74 GB27.0 GB
NVFP4cuda17.5 GB10.74 GB31.6 GB

KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.

Benchmarks

MMLU-Pro
33
IFEval
68
Arena ELO
1150

GPUs that run Command-R 35B natively (47)

Plus 36 GPUs that run it with CPU offload (slower)

Notes

Full attention (no GQA) — heavy KV cache at long context.

Hugging Face ↗Ollama ↗Released 2024-08-30

Compare Command-R 35B with other models

How to run Command-R 35B locally

816244880160320

Q4_K_M needs 35.9 GB — needs a workstation or datacenter GPU (48–80 GB).

Ollama

ollama run command-r:35b

llama.cpp

./llama-cli -m c4ai-command-r-35b.Q4_K_M.gguf -c 8192 -ngl 99

LM Studio: Search for 'Command R 35B' in LM Studio. Note the heavy KV cache at long context due to full attention -- keep context under 32K for best performance.

Why this quantization? Command R has 35B dense parameters with full attention (64 KV heads, no GQA), which makes its KV cache substantially larger than other models of similar size. Q4_K_M is essential for keeping the weight footprint manageable (roughly 20 GB), leaving VRAM budget for the unusually heavy KV cache. Users should be aware that the full-attention design limits practical context length compared to GQA models.

Who is Command-R 35B for?

Developers building RAG (Retrieval Augmented Generation) pipelines who want a model specifically designed for grounded, citation-backed responses. Also strong for multilingual applications. Note the CC-BY-NC license restricts commercial use.

Best for

  • RAG pipelines with citation generation and source attribution
  • Multilingual document processing and question answering
  • Enterprise search and knowledge base applications (non-commercial)
  • Grounded responses that cite their sources accurately

Not ideal for

  • Commercial deployments -- the CC-BY-NC 4.0 license prohibits commercial use
  • Long-context tasks -- the full-attention architecture makes the KV cache very heavy beyond 32K tokens
  • General chat or creative writing -- other 32B-class models outperform on general benchmarks

Continue reading

vram-guides9 min
Best LLMs for 48 GB VRAM (2026)

Frequently asked questions

What are the VRAM requirements for Command-R 35B?
Command-R 35B requires approximately 35.9 GB of VRAM at Q4_K_M quantization, 53.7 GB at Q8, and 90.4 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
How many parameters does Command-R 35B have?
Command-R 35B has 35 billion parameters.
How capable is Command-R 35B?
Command-R 35B has an MMLU-Pro score of 33, making it well-suited for lightweight tasks, prototyping, and resource-constrained environments.
Can Command-R 35B run on a 16 GB GPU?
No. At Q4_K_M, Command-R 35B needs 35.9 GB of VRAM — more than 16 GB. You will need a 48 GB GPU like the RTX 6000 Ada or a dual-GPU setup.
Can Command-R 35B run on a 24 GB GPU?
No. Even at Q4_K_M, Command-R 35B needs 35.9 GB. Consider a 48 GB card like the RTX 6000 Ada or a dual RTX 4090 setup.
What is the smallest quantization for Command-R 35B that fits in 24 GB of VRAM?
Command-R 35B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 27.0 GB at Q2_K.
What GPU do I need to run Command-R 35B locally?
You need a 48 GB GPU or a dual-GPU setup. At Q4_K_M, Command-R 35B needs 35.9 GB VRAM. Options: RTX 6000 Ada (48 GB), A6000 (48 GB), or 2× RTX 4090.