About CanItRun

How CanItRun decides whether a model fits on your hardware, and where the data comes from.

What is CanItRun?

CanItRun is a free, open calculator for developers running large language models locally. You pick your GPU — from a laptop-class RTX 4060 to a datacenter H100 or an Apple M3 Max — and the tool instantly shows which open-weight models will fit in VRAM, at what quantization level, and roughly how many tokens per second to expect. Benchmark scores (MMLU-Pro, GPQA, Arena ELO) are shown alongside each model so you can weigh performance tradeoffs at a glance.

The calculations are based on published architecture specs from Hugging Face model cards, observed GGUF file sizes from the Ollama library, and GPU specs from manufacturer datasheets. No proprietary data, no pay-to-rank — every verdict is the same regardless of hardware vendor.

When a model needs more VRAM than your hardware has, we show a RunPod affiliate link. RunPod is a cloud GPU rental service — you rent a datacenter GPU by the hour instead of buying one, with no contract, and can cancel anytime; sessions typically start in about a minute. It's clearly marked and never changes what fits, what quantization is recommended, or how a model is ranked. If you sign up through our link, CanItRun may earn a commission.

Verdicts

Fits — total memory ≤ available VRAM. Runs at full GPU speed.
Offload— weights spill into system RAM via CPU offload. Usable but slower. We don't apply a flat penalty: the tok/s estimate scales with exactly how much of the model spilled out of VRAM, so a model that's 90% in VRAM is estimated much closer to full speed than one that's only 20% in VRAM. Whatever fraction spilled falls back to an assumed 50 GB/s system-RAM bandwidth, well below any GPU's VRAM bandwidth. Mixture-of-experts models are estimated the way llama.cpp's --n-cpu-moe works: attention, shared weights and the KV cache stay on the GPU, and only the experts a token actually routes to are read from system RAM, which is far faster than spilling the whole model uniformly.
Won't run — even with RAM offload, the model won't fit.

Tokens per second estimate

Local LLM inference is memory-bandwidth-bound, not FLOPS-bound: generating each token means reading every active weight and the entire KV cache off memory once. Our estimate accounts for both terms, not just the weights:

seconds per token ≈ (active_weights_GB + kv_cache_GB) / (memory_bandwidth_GBs × decode_efficiency)
                + layers × per_layer_overhead

tok/s = 1 / (seconds per token)

Folding in the KV cache is what makes the estimate drop as context grows, not just as the model gets bigger — the same model estimates meaningfully slower at 64K context than at 4K, because there's more cache to read on every step.

decode_efficiency accounts for real hardware never sustaining 100% of its rated bandwidth for this workload (on-the-fly dequantization, imperfect memory access): 88% for discrete GPUs (CUDA/ROCm/Vulkan), 80% for Apple Silicon's unified memory, and 50% for plain system RAM. The per_layer_overheadterm is the fixed cost of launching each layer's kernels, about 60 microseconds per layer on a discrete GPU. It is why a 70B model streams at a higher share of rated bandwidth than an 8B one, and why several RTX 4090s running one 70B model together are estimated at about the speed of one card at the same bytes, not four times faster. These are calibrated against public llama.cpp benchmarks, not assumed: see the code comments in lib/calc.ts and the anchor table in lib/calc-speed-anchors.test.ts for the specific measurements behind each figure.

For MoE models, we use active parameters for the weights term (only the experts activated per token count toward bandwidth) and add a routing cost per layer, because the small indexed expert matmuls are much less efficient than dense ones. That cost is small on NVIDIA CUDA and several times larger on AMD (ROCm/Vulkan) and Apple Metal, matching published GPT-OSS results on each. We also add 12% on top of the weights-plus-KV total for activation memory and runtime overhead the raw arithmetic doesn't capture. Multi-GPU estimates assume llama.cpp's default layer split, which runs the cards one after another, so extra cards add capacity, not speed; tensor-parallel modes and vLLM can do better. Real-world throughput varies ±30% with quantization kernels, batch size, and runtime (llama.cpp, vLLM, MLX). Treat these as ballpark, not promises.

How much VRAM is actually usable

A GPU's nominal VRAM isn't all available to model weights — some is always held back for driver and runtime overhead, and on Apple Silicon, system RAM and VRAM are the same physical pool. We show this reserve explicitly rather than silently assuming the full nominal capacity is free:

Single discrete GPU: 5% of nominal VRAM is held back. A 24 GB card has roughly 22.8 GB usable, not 24 GB — a model whose real total lands just under 24 GB can still need CPU offload once this is applied.
Multiple pooled discrete GPUs: 10% of the combined total is held back, on top of PCIe and fragmentation overhead from splitting a model across cards.
Apple Silicon (unified memory): a fixed 8 GB is reserved for macOS and background apps — not a percentage of total memory. That's a small fraction of a 128 GB chip's pool, but a large one on a 16 GB chip.

When a model's real download size beats the formula

Our default quantization ladder estimates file size as parameters × bytes-per-weight, calibrated against typical real GGUF files. Some releases don't follow that pattern — GPT-OSS 120B, for example, ships its MoE expert tensors only in native MXFP4, so real downloads across every supported quant level cluster within a few GB of each other (roughly 62.6–65.4 GB) instead of scaling down the way the generic ladder assumes. For models like this, we override the formula with a documented floor sourced from the real published file sizes, so the table matches what you'd actually download rather than a number the ladder arithmetic implies but no real file matches.

Data sources

Model metadata (params, context, license, architecture): Hugging Face model cards.
Quantized file sizes: Ollama library GGUF builds.
Benchmarks (MMLU-Pro, GPQA, IFEval, MATH, BBH, MuSR): Open LLM Leaderboard v2.
Arena ELO: LMSYS Chatbot Arena.
GPU specs (VRAM, memory bandwidth): manufacturer datasheets and TechPowerUp.

Feedback & contact

Found a bug, have a correction, or want to suggest a GPU or model we're missing? Email us at [email protected].

Limitations

Inference only — training and fine-tuning need roughly 4–6× more memory.
Batch size 1, single-user chat. Concurrent users need proportionally more KV cache.
No speculative decoding or flash-attention discounts applied.
Apple Silicon unified memory: we reserve a fixed 8 GB for macOS and background apps, not a percentage of total RAM — see “How much VRAM is actually usable” above.