About CanItRun

What is CanItRun?

CanItRun is a free, open calculator for developers running large language models locally. You pick your GPU — from a laptop-class RTX 4060 to a datacenter H100 or an Apple M3 Max — and the tool instantly shows which open-weight models will fit in VRAM, at what quantization level, and roughly how many tokens per second to expect. Benchmark scores (MMLU-Pro, GPQA, Arena ELO) are shown alongside each model so you can weigh performance tradeoffs at a glance.

The calculations are based on published architecture specs from Hugging Face model cards, observed GGUF file sizes from the Ollama library, and GPU specs from manufacturer datasheets. No proprietary data, no pay-to-rank — every verdict is the same regardless of hardware vendor.

When a model needs more VRAM than your hardware has, we show a RunPod affiliate link. RunPod is a cloud GPU rental service — you rent a datacenter GPU by the hour instead of buying one, with no contract, and can cancel anytime; sessions typically start in about a minute. It's clearly marked and never changes what fits, what quantization is recommended, or how a model is ranked. If you sign up through our link, CanItRun may earn a commission.

Verdicts

• Fits — total memory ≤ available VRAM. Runs at full GPU speed.
• Offload— weights spill into system RAM via CPU offload. Usable but slower. We don't apply a flat penalty: the tok/s estimate scales with exactly how much of the model spilled out of VRAM, so a model that's 90% in VRAM is estimated much closer to full speed than one that's only 20% in VRAM. Whatever fraction spilled falls back to an assumed 50 GB/s system-RAM bandwidth, well below any GPU's VRAM bandwidth.
• Won't run — even with RAM offload, the model won't fit.

Tokens per second estimate

Local LLM inference is memory-bandwidth-bound, not FLOPS-bound: generating each token means reading every active weight and the entire KV cache off memory once. Our estimate accounts for both terms, not just the weights:

tok/s ≈ (memory_bandwidth_GBs × decode_efficiency) / (active_weights_GB + kv_cache_GB)

Folding in the KV cache is what makes the estimate drop as context grows, not just as the model gets bigger — the same model estimates meaningfully slower at 64K context than at 4K, because there's more cache to read on every step.

decode_efficiency accounts for real hardware never sustaining 100% of its rated bandwidth for this workload (kernel launch overhead, attention/softmax compute, on-the-fly dequantization): 65% for discrete GPUs (CUDA/ROCm/Vulkan), 80% for Apple Silicon's unified memory, which tracks measurably closer to its rated bandwidth, and 50% for plain system RAM. These are calibrated against public benchmarks, not assumed — see the code comments in lib/calc.ts for the specific numbers behind each figure.

For MoE models, we use active parameters for the weights term (only the experts activated per token count toward bandwidth), further discounted for real-world routing and small per-expert batch=1 matmul overhead, which eats into the naive bandwidth advantage active-only weights would otherwise suggest. We also add 12% on top of the weights-plus-KV total for activation memory and runtime overhead the raw arithmetic doesn't capture. Real-world throughput varies ±30% with quantization kernels, batch size, and runtime (llama.cpp, vLLM, MLX). Treat these as ballpark, not promises.

How much VRAM is actually usable

A GPU's nominal VRAM isn't all available to model weights — some is always held back for driver and runtime overhead, and on Apple Silicon, system RAM and VRAM are the same physical pool. We show this reserve explicitly rather than silently assuming the full nominal capacity is free:

• Single discrete GPU: 5% of nominal VRAM is held back. A 24 GB card has roughly 22.8 GB usable, not 24 GB — a model whose real total lands just under 24 GB can still need CPU offload once this is applied.
• Multiple pooled discrete GPUs: 10% of the combined total is held back, on top of PCIe and fragmentation overhead from splitting a model across cards.
• Apple Silicon (unified memory): a fixed 8 GB is reserved for macOS and background apps — not a percentage of total memory. That's a small fraction of a 128 GB chip's pool, but a large one on a 16 GB chip.

When a model's real download size beats the formula

Our default quantization ladder estimates file size as parameters × bytes-per-weight, calibrated against typical real GGUF files. Some releases don't follow that pattern — GPT-OSS 120B, for example, ships its MoE expert tensors only in native MXFP4, so real downloads across every supported quant level cluster within a few GB of each other (roughly 62.6–65.4 GB) instead of scaling down the way the generic ladder assumes. For models like this, we override the formula with a documented floor sourced from the real published file sizes, so the table matches what you'd actually download rather than a number the ladder arithmetic implies but no real file matches.

Data sources

• Model metadata (params, context, license, architecture): Hugging Face model cards.
• Quantized file sizes: Ollama library GGUF builds.
• Benchmarks (MMLU-Pro, GPQA, IFEval, MATH, BBH, MuSR): Open LLM Leaderboard v2.
• Arena ELO: LMSYS Chatbot Arena.
• GPU specs (VRAM, memory bandwidth): manufacturer datasheets and TechPowerUp.

Feedback & contact

Found a bug, have a correction, or want to suggest a GPU or model we're missing? Email us at [email protected].

Limitations

• Inference only — training and fine-tuning need roughly 4–6× more memory.
• Batch size 1, single-user chat. Concurrent users need proportionally more KV cache.
• No speculative decoding or flash-attention discounts applied.
• Apple Silicon unified memory: we reserve a fixed 8 GB for macOS and background apps, not a percentage of total RAM — see “How much VRAM is actually usable” above.