Run LLMs on Your Own Hardware
Find which open-weight models fit your GPU, learn how to set them up, and discover the best tools for local AI — all in one place.
I have a GPU — what can I run? ↓
See which open-weight LLMs fit your GPU — with quantization levels, benchmarks, and tokens/sec estimates.
I want to choose a GPU →
Compare VRAM, bandwidth, and real-world performance across GPUs and Apple Silicon.
I'm new to local LLMs →
Quantization, Ollama, VRAM vs RAM — everything to run your first model in 10 minutes.
Already have a GPU? Pick it below to see which models fit — at which quantization, with benchmarks and estimated performance.
- Q2_K81.4GLM-4.5 Air 106BZ.ai · 106B (12B active) · reasoning
- Q2_K79.9GLM-4.6V 106BZ.ai · 106B (12B active)
- NVFP471.1Qwen 2.5 72B InstructAlibaba · 72B
Columns: verdict · model · best quant · headline benchmark · est. tokens/sec. Hover the score for its benchmark. Click a row for the full breakdown.
Featured Guides
Practical guides for running open-weight LLMs on your hardware — from choosing the right quantization format to finding the best model for your VRAM budget.
GGUF Quantization Explained: Q4, Q5, Q6, Q8 Compared
Quantization is the single most important technique for running large models on consumer hardware. Here is how each GGUF quantization level actually works and when to use it.
June 28, 2026Best LLMs You Can Run on 8 GB VRAM (2026)
Eight gigabytes of VRAM is tight but far from useless. Here are the best models that actually fit, with honest trade-offs and practical setup advice.
June 28, 2026Best GPU for Running LLMs Locally in 2026
Buying a GPU for local LLMs? VRAM matters more than compute. Here is a practical buyer's guide covering every budget tier from $200 to $2000 and beyond.
June 28, 2026Getting Started with Ollama: Run Any LLM in One Command
Ollama makes running LLMs locally as simple as a single terminal command. Here is everything you need from installation to advanced customization.
June 28, 2026Best LLMs for 16 GB VRAM GPUs (2026)
Sixteen gigabytes of VRAM is the sweet spot for local LLM inference in 2026. Here are the models that make the most of it across coding, reasoning, and general use.
July 20, 2026How Much VRAM Does Llama 3 Need? Complete Guide
Find out exactly how much VRAM you need to run Llama 3 models locally, from the 8B variant on a budget GPU to the full 405B on multi-GPU setups.
June 28, 2026Trending Models
The most popular open-weight models right now — see how they compare and which GPUs can run them.
Popular GPUs for Local LLMs
These GPUs are the most popular choices for local LLM inference — from consumer cards to Apple Silicon.
Run AI apps with the right model
Not every AI app needs the same hardware. Coding agents, chat frontends, roleplay tools, and self-hosted apps can use local models, OpenRouter models, or both. Find which apps work with your setup — and which models make them useful.
Coding Agents
Cline, Roo Code, Aider, Continue, Claude Code — find which works with your hardware.
Chat Frontends
Open WebUI, LibreChat, SillyTavern — self-hosted or cloud, find your setup.
Local LLM Tools
Ollama, LM Studio, llama.cpp, vLLM — pick the right engine for your GPU.
Self-Hosted Apps
Open WebUI, LibreChat, SillyTavern — run AI apps on your own hardware with full privacy.
Why This Tool Exists
Running large language models locally gives you privacy, control, and zero inference costs — but figuring out which models fit on your GPU is a manual, error-prone process. CanItRun eliminates the guesswork. We built this as a free, open tool for developers, researchers, and hobbyists who want to experiment with open-weight LLMs without cloud dependencies.
How it works:Model VRAM requirements are calculated from three components: base model weights (adjusted for quantization level), key-value cache for your target context length, and activation memory for inference. We then compare these requirements against a comprehensive database of real GPU specifications and community-reported benchmarks to tell you not just whether a model fits, but how it's likely to perform.
Who should use this:ML engineers prototyping locally, researchers on academic budgets, students learning about LLMs, and hobbyists running models on consumer hardware. If you're evaluating whether to upgrade your GPU or trying to squeeze the largest model onto your existing setup, this tool helps you make data-driven decisions.
What sets CanItRun apart: Unlike generic calculators, we maintain an extensive GPU database with real-world benchmarks, provide quantization-specific recommendations (from FP16 down to INT4), and show expected tokens-per-second performance based on community data. We also track emerging architectures and new GPU releases to keep recommendations current.
Common Use Cases
Local Development & Prototyping
Test prompts, fine-tune adapters, and iterate on RAG pipelines without incurring API costs or sending proprietary data to the cloud.
Privacy-Sensitive Applications
Process legal documents, medical records, or internal communications with zero data leaving your infrastructure.
Education & Research
Students and researchers can experiment with state-of-the-art models on academic budgets using consumer or lab hardware.
Edge Deployment
Evaluate which models fit on target deployment hardware — from Jetson Orin to Mac mini to gaming laptops with limited VRAM.