Kimi K3
Kimi K3 needs roughly 1910.3 GB VRAM at Q4_K_M quantization (6272.4 GB at FP16). 0 GPUs we track can run it fully in VRAM at 8k context.
0 GPUs run this natively · 0 with CPU offload
Kimi K3 is a Mixture of Experts (MoE) model with 2800B total parameters but only 104B active per token developed by Moonshot AI. Announced July 16, 2026 and opened as downloadable weights on July 27, it is the largest open-weight model anyone has published. The architecture breaks from the uniform MLA stack Moonshot used through K2.5: 93 layers interleave 69 Kimi Delta Attention layers — a linear-attention variant Moonshot credits with up to 6.3x faster decoding at long context — with 24 gated MLA layers, held together by Attention Residuals and a Stable LatentMoE router that fires 16 of 896 experts per token. A 401M-parameter MoonViT-V2 encoder takes images and video, the 1M-token window is native rather than bolted on after training, and thinking is always enabled with low/high/max effort settings rather than a separate reasoning mode you toggle.
To run Kimi K3 locally: there is no consumer path, and no quantization trick that creates one. The MXFP4 checkpoint is roughly 1.4 TB of weights before KV cache or activations, so the floor is terabytes of aggregate accelerator memory — realistically eight or more nodes of 8x80GB, with Moonshot's own guidance pointing at a supernode of 64+ accelerators. Unlike a 70B model, CPU offload doesn't rescue it: spilling even a fraction of that weight pool to system RAM leaves throughput unusable. What the hybrid attention does buy you is context headroom — only the 24 gated-MLA layers accumulate a KV cache, so a 1M-token session costs a fraction of what 93 uniform-attention layers would, and long-context work is bounded by the weights rather than by the cache. For everyone without a cluster, the API or a hosted endpoint is the realistic route. As a MoE model, inference speed depends on active parameters (104B) rather than total size.
GPQA-Diamond 93.5 and Terminal-Bench 2.1 88.3 put K3 at the front of the open-weight field, and it takes SWE-bench Verified to 76.8 with first-place finishes on SWE-Marathon and Program Bench — sustained multi-hour agent sessions are where it pulls away from K2.5, more than single-shot puzzles. Read the coding numbers with the harness in mind: Moonshot ran them through its own KimiCode agent, and harness choice alone moves published scores by 10-26 points. Independent measurement lands close but not identical, with Artificial Analysis reporting GPQA 94 and Terminal-Bench 85. The number that deserves the most attention is not a win: on AA-Omniscience, accuracy climbed to 46% while the hallucination rate climbed alongside it to 51%, so K3 attempts more questions and gets more of them confidently wrong. Pair it with retrieval or verification for factual work, and lean on it where an agent's output is checkable — code that compiles, commands that either succeed or don't.
VRAM at each quantization
Calculated at 8k context. Since KV cache scales linearly with context, longer sessions need more VRAM than shown here.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 11200.0 GB | 0.40 GB | 12544.5 GB |
| BF16 | 5600.0 GB | 0.40 GB | 6272.4 GB |
| FP16 | 5600.0 GB | 0.40 GB | 6272.4 GB |
| Q8_0 | 2976.4 GB | 0.40 GB | 3334.0 GB |
| Q6_K | 2298.8 GB | 0.40 GB | 2575.1 GB |
| Q5_K_M | 1993.6 GB | 0.40 GB | 2233.3 GB |
| Q4_K_M | 1705.2 GB | 0.40 GB | 1910.3 GB |
| Q3_K_M | 1346.8 GB | 0.40 GB | 1508.9 GB |
| Q2_K | 1066.8 GB | 0.40 GB | 1195.3 GB |
| NVFP4reccuda | 1400.0 GB | 0.40 GB | 1568.5 GB |
KV cache figures assume 8k context at FP16. NVFP4 quantization requires a CUDA-capable GPU. Enable TurboQuant in the calculator to see reduced KV cache estimates.
Benchmarks
GPUs that run Kimi K3 natively (0)
No single GPU in our list fits this model at Q4 with 8k context. Browse all GPUs or try multi-GPU setups.
Notes
The largest open-weight model published to date: 896 experts with 16 routed plus 2 shared per token (104B activated), 93 layers split 69 Kimi Delta Attention / 24 gated MLA, Attention Residuals, a Stable LatentMoE router, a 401M-parameter MoonViT-V2 vision encoder, and a native 1M-token window. Thinking is always on, with low/high/max effort levels. The API launched July 16, 2026 and the weights opened July 27 under the custom Kimi K3 License — MIT-like, but model-as-a-service businesses past $20M revenue over any 12 months need a separate agreement with Moonshot, and products past 100M MAU must display "Kimi K3" in their interface. The checkpoint is MXFP4 weights with MXFP8 activations, quantization-aware trained from the SFT stage onward, so the ~1.4 TB 4-bit row is the real download rather than a lossy community conversion; the higher-precision rows in the quantization table are arithmetic, not published checkpoints. Benchmark figures are Moonshot-reported through its own KimiCode harness — swapping harnesses moves published coding scores by 10-26 points, and Artificial Analysis independently measures GPQA-Diamond 94 and Terminal-Bench 2.1 85. Moonshot does not publish MMLU-Pro or LiveCodeBench for K3.
Compare Kimi K3 with other models
How to run Kimi K3 locally
NVFP4 needs 1568.5 GB — needs multiple datacenter-class GPUs (80 GB+ each).
vLLM (MXFP4, multi-node)
vllm serve moonshotai/Kimi-K3 --data-parallel-size 16 --enable-expert-parallel --trust-remote-codeLM Studio: There is no GGUF build and no local Ollama tag — desktop frontends can only point at a remote endpoint. Moonshot contributed Kimi Delta Attention kernels with prefix caching to vLLM and SGLang; other runtimes need patches for the 896-expert router before they will load the checkpoint at all.
Why this quantization? The 4-bit row is the model, not a compromise. Moonshot quantization-aware trained K3 to MXFP4 weights with MXFP8 activations from the SFT stage onward and ships nothing higher, so the roughly 1.4 TB figure is the actual download and the FP16/BF16 rows above it are arithmetic rather than checkpoints you can obtain. Blackwell and MI400 accelerators run MXFP4 natively; older hardware pays a dequantization cost on every expert read.
Who is Kimi K3 for?
Organizations with a multi-node GPU cluster that want a frontier agentic model inside their own network — typically for data-residency reasons, since the same weights back Moonshot's API at a fraction of the operational effort.
Best for
- Long-horizon coding agents that run for hours across a repository
- Million-token context work — whole codebases, filings, or long transcripts without retrieval
- Screenshot-, diagram-, and video-grounded engineering tasks
- Self-hosted deployments where prompts cannot leave your infrastructure
Not ideal for
- Any single-GPU or single-node machine — the weights alone exceed a terabyte
- CPU offload setups, which collapse to unusable throughput at this scale
- Factual lookup without verification, given the elevated hallucination rate that came with K3's accuracy gains
- Teams that just want the capability — the API costs far less than standing up 64 accelerators
Frequently asked questions
- What are the VRAM requirements for Kimi K3?
- Kimi K3 requires approximately 1910.3 GB of VRAM at Q4_K_M quantization, 3334.0 GB at Q8, and 6272.5 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Kimi K3 have?
- Kimi K3 has 2800 billion total parameters, but only 104 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- Can Kimi K3 run on a 16 GB GPU?
- No. At Q4_K_M, Kimi K3 needs 1910.3 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Kimi K3 run on a 24 GB GPU?
- No. Even at Q4_K_M, Kimi K3 needs 1910.3 GB. Consider a multi-GPU server with 80 GB+ total VRAM.
- What is the smallest quantization for Kimi K3 that fits in 24 GB of VRAM?
- Kimi K3 cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 1195.3 GB at Q2_K.
- What GPU do I need to run Kimi K3 locally?
- You need a multi-GPU server. At Q4_K_M, Kimi K3 needs 1910.3 GB VRAM, more than any single consumer GPU. Consider 2–4× H100 or A100 GPUs.