Phi-4-mini Instruct
Phi-4-mini Instruct needs roughly 3.8 GB VRAM at Q4_K_M quantization (9.7 GB at FP16). 117 GPUs we track can run it fully in VRAM at 8k context.
117 GPUs run this natively · 1 with CPU offload
Phi-4-mini Instruct is a 3.8B parameter dense large language model developed by Microsoft. Released in February 2025, it is a text-only model with a 128K context window, released under the MIT license, allowing commercial use. Same size class as Phi-3.5 Mini but adds GQA (8 KV heads, versus Phi-3.5's full 32) for a much smaller KV cache, plus function calling and a 200K vocabulary.
To run Phi-4-mini Instruct locally, you need approximately 3.8 GB of VRAM at Q4_K_M quantization with 8k context. 117 of the GPUs we track can run it fully in VRAM, with a further 1 able to offload to system RAM. At Q4_K_M it needs just 3.8 GB, making it accessible even on mid-range 16 GB cards like RTX 4080 and RTX 5070 Ti. At Q8_K_M (5.7 GB), you get near-FP16 quality while still fitting on 8, 12, 16, 24, 32, 48 and 80 GB GPUs. FP16 requires 9.7 GB, limiting it to 48 and 80 GB GPUs.
With an MMLU-Pro score of 67.3, it delivers strong general reasoning for local deployment. The license allows commercial use.
VRAM at each quantization
Figures below assume 8k context; KV cache grows linearly as context length increases.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 15.2 GB | 1.07 GB | 18.2 GB |
| BF16 | 7.6 GB | 1.07 GB | 9.7 GB |
| FP16 | 7.6 GB | 1.07 GB | 9.7 GB |
| Q8_0 | 4.0 GB | 1.07 GB | 5.7 GB |
| Q6_Krec | 3.1 GB | 1.07 GB | 4.7 GB |
| Q5_K_M | 2.7 GB | 1.07 GB | 4.2 GB |
| Q4_K_M | 2.3 GB | 1.07 GB | 3.8 GB |
| Q3_K_M | 1.8 GB | 1.07 GB | 3.3 GB |
| Q2_K | 1.4 GB | 1.07 GB | 2.8 GB |
| NVFP4cuda | 1.9 GB | 1.07 GB | 3.3 GB |
Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.
Benchmarks
GPUs that run Phi-4-mini Instruct natively (117)
- NVIDIA RTX 5090BF16 · 134.8 t/s
- NVIDIA RTX 5080BF16 · 82.1 t/s
- NVIDIA RTX 5070 TiBF16 · 77.4 t/s
- NVIDIA RTX 5070BF16 · 60.3 t/s
- NVIDIA RTX 5060 Ti 16GBBF16 · 41.8 t/s
Show 112 more
- NVIDIA RTX 5060 Ti 8GBNVFP4 · 105.7 t/s
- NVIDIA RTX 5060NVFP4 · 105.7 t/s
- NVIDIA RTX 5050NVFP4 · 80.1 t/s
- NVIDIA RTX 4090BF16 · 85.5 t/s
- NVIDIA RTX 4080BF16 · 63.8 t/s
- NVIDIA RTX 4070 Ti SUPERBF16 · 60.3 t/s
- NVIDIA RTX 4070 TiBF16 · 46.6 t/s
- NVIDIA RTX 4070 SUPERBF16 · 46.6 t/s
- NVIDIA RTX 4070BF16 · 46.6 t/s
- NVIDIA RTX 4060 Ti 16GBBF16 · 27.7 t/s
- NVIDIA RTX 4060Q8_0 · 43 t/s
- NVIDIA RTX 3090BF16 · 80.3 t/s
- NVIDIA RTX 3090 TiBF16 · 85.5 t/s
- NVIDIA RTX 3080 10GBQ8_0 · 104.5 t/s
- NVIDIA RTX 3060 12GBBF16 · 34.1 t/s
- NVIDIA B300 288GBBF16 · 154.1 t/s
- NVIDIA B200 180GBBF16 · 154.1 t/s
- NVIDIA H200 141GBBF16 · 136.8 t/s
- NVIDIA H100 80GBBF16 · 122 t/s
- NVIDIA A100 80GBBF16 · 99.1 t/s
- NVIDIA A100 40GBBF16 · 86.2 t/s
- NVIDIA L40SBF16 · 60 t/s
- NVIDIA RTX A6000BF16 · 67.8 t/s
- NVIDIA RTX 4000 AdaBF16 · 30.6 t/s
- NVIDIA RTX 4500 AdaBF16 · 40.4 t/s
- NVIDIA RTX 5000 AdaBF16 · 52.5 t/s
- NVIDIA RTX 6000 AdaBF16 · 82.1 t/s
- NVIDIA RTX Pro 6000BF16 · 134.8 t/s
- NVIDIA DGX Spark (128GB)BF16 · 26.3 t/s
- AMD Radeon RX 9070 XTBF16 · 57.7 t/s
- AMD Radeon RX 7900 XTXBF16 · 82.1 t/s
- AMD Radeon RX 7900 XTBF16 · 70.2 t/s
- AMD Radeon RX 7900 GREBF16 · 52.5 t/s
- AMD Radeon RX 7800 XTBF16 · 56.4 t/s
- AMD Radeon RX 6800 XTBF16 · 47.2 t/s
- AMD Radeon PRO W7800BF16 · 52.5 t/s
- AMD Radeon PRO W7900BF16 · 75 t/s
- AMD Instinct MI300XBF16 · 140.5 t/s
- AMD Radeon AI PRO R9700 32GBBF16 · 57.7 t/s
- AMD Strix Halo (128GB)BF16 · 24.7 t/s
- AMD Strix Halo (96GB)BF16 · 24.7 t/s
- AMD Strix Halo (64GB)BF16 · 24.7 t/s
- AMD Strix Halo (32GB)BF16 · 24.7 t/s
- Apple M5 Ultra (512GB)BF16 · 110.7 t/s
- Apple M5 Ultra (256GB)BF16 · 110.7 t/s
- Apple M5 Ultra (96GB)BF16 · 110.7 t/s
- Apple M5 Max (128GB)BF16 · 56.6 t/s
- Apple M5 Max (64GB)BF16 · 56.6 t/s
- Apple M5 Max (48GB)BF16 · 56.6 t/s
- Apple M5 Max (36GB)BF16 · 42.4 t/s
- Apple M5 Pro (64GB)BF16 · 28.3 t/s
- Apple M5 Pro (48GB)BF16 · 28.3 t/s
- Apple M5 Pro (24GB)BF16 · 28.3 t/s
- Apple M5 (32GB)BF16 · 14.1 t/s
- Apple M5 (16GB)Q8_0 · 23.9 t/s
- Apple M6 (32GB)BF16 · 15.7 t/s
- Apple M6 (16GB)Q8_0 · 26.6 t/s
- Apple M4 Max (128GB)BF16 · 50.4 t/s
- Apple M4 Max (64GB)BF16 · 50.4 t/s
- Apple M4 Max (48GB)BF16 · 50.4 t/s
- Apple M4 Max (36GB)BF16 · 37.8 t/s
- Apple M4 Pro (48GB)BF16 · 25.2 t/s
- Apple M4 Pro (24GB)BF16 · 25.2 t/s
- Apple M4 (32GB)BF16 · 11.1 t/s
- Apple M4 (16GB)Q8_0 · 18.8 t/s
- Apple M3 Ultra (512GB)BF16 · 75.5 t/s
- Apple M3 Ultra (256GB)BF16 · 75.5 t/s
- Apple M3 Ultra (96GB)BF16 · 75.5 t/s
- Apple M3 Max (128GB)BF16 · 36.9 t/s
- Apple M3 Max (96GB)BF16 · 27.7 t/s
- Apple M3 Max (64GB)BF16 · 36.9 t/s
- Apple M3 Max (48GB)BF16 · 36.9 t/s
- Apple M3 Max (36GB)BF16 · 27.7 t/s
- Apple M3 Pro (36GB)BF16 · 13.8 t/s
- Apple M3 Pro (18GB)BF16 · 13.8 t/s
- Apple M3 (24GB)BF16 · 9.2 t/s
- Apple M3 (16GB)Q8_0 · 15.6 t/s
- Apple M2 Ultra (192GB)BF16 · 73.8 t/s
- Apple M2 Ultra (64GB)BF16 · 73.8 t/s
- Apple M2 Max (96GB)BF16 · 36.9 t/s
- Apple M2 Max (64GB)BF16 · 36.9 t/s
- Apple M2 Max (32GB)BF16 · 36.9 t/s
- Apple M2 Pro (32GB)BF16 · 18.4 t/s
- Apple M2 Pro (16GB)Q8_0 · 31.3 t/s
- Apple M2 (24GB)BF16 · 9.2 t/s
- Apple M2 (16GB)Q8_0 · 15.6 t/s
- Apple M1 Ultra (128GB)BF16 · 73.8 t/s
- Apple M1 Ultra (64GB)BF16 · 73.8 t/s
- Apple M1 Max (64GB)BF16 · 36.9 t/s
- Apple M1 Max (32GB)BF16 · 36.9 t/s
- Apple M1 Pro (32GB)BF16 · 18.4 t/s
- Apple M1 Pro (16GB)Q8_0 · 31.3 t/s
- Apple M1 (16GB)Q8_0 · 10.6 t/s
- Intel Arc B580 12GBBF16 · 42.5 t/s
- Intel Arc B570 10GBQ8_0 · 58.1 t/s
- Intel Arc Pro B70 32GBBF16 · 55.2 t/s
- Intel Arc Pro B60 24GBBF16 · 35.9 t/s
- Intel Arc Pro B50 16GBBF16 · 21.8 t/s
- Intel Arc A770 16GBBF16 · 51.2 t/s
- Intel Arc A770 8GBQ8_0 · 75.4 t/s
- Intel Arc A750 8GBQ8_0 · 75.4 t/s
- Intel Arc A580 8GBQ8_0 · 75.4 t/s
- Intel Arc A380 6GBQ6_K · 36.3 t/s
- Intel Arc A310 4GBQ4_K_M · 30.3 t/s
- Intel Arc Pro A60 12GBBF16 · 36.2 t/s
- Intel Arc Pro A50 6GBQ6_K · 37.4 t/s
- Intel Arc Pro A40 6GBQ6_K · 37.4 t/s
- Intel Data Center GPU Max 1550BF16 · 121 t/s
- Intel Data Center GPU Max 1100BF16 · 75.3 t/s
- Intel Arc 140V (32GB)BF16 · 13.5 t/s
- Intel Arc 140V (16GB)Q8_0 · 22.6 t/s
- Intel Arc 130V (16GB)Q8_0 · 22.6 t/s
Plus 1 GPU that runs it with CPU offload (slower)
- CPU only (system RAM)BF16 · 4.6 t/s
No GPU? Rent an RTX 3080 10GB from $0.17/hr ↗
Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.
Notes
Same size class as Phi-3.5 Mini but adds GQA (8 KV heads, versus Phi-3.5's full 32) for a much smaller KV cache, plus function calling and a 200K vocabulary.
Continue reading
Frequently asked questions
- What are the VRAM requirements for Phi-4-mini Instruct?
- Phi-4-mini Instruct requires approximately 3.8 GB of VRAM at Q4_K_M quantization, 5.7 GB at Q8, and 9.7 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Phi-4-mini Instruct have?
- Phi-4-mini Instruct has 3.8 billion parameters.
- How capable is Phi-4-mini Instruct?
- With an MMLU-Pro score of 67.3, Phi-4-mini Instruct delivers solid general-purpose performance suitable for most everyday tasks and professional use.
- Can Phi-4-mini Instruct run on a 16 GB GPU?
- Yes. Phi-4-mini Instruct needs 3.8 GB at Q4_K_M, which fits in a 16 GB GPU like the RTX 4080 or RTX 5070 Ti.
- What is the smallest quantization for Phi-4-mini Instruct that fits in 24 GB of VRAM?
- At BF16, Phi-4-mini Instruct needs 9.7 GB, the highest-quality quantization that fits in 24 GB of VRAM.
- What GPU do I need to run Phi-4-mini Instruct locally?
- A 16 GB GPU is enough. At Q4_K_M, Phi-4-mini Instruct needs 3.8 GB VRAM. Good options: RTX 4080 (16 GB), RTX 5070 Ti (16 GB).
- Can I run Phi-4-mini Instruct in the cloud?
- Yes. The cheapest RunPod setup we found that holds Phi-4-mini Instruct entirely in GPU memory at 8k context is 1× RTX 3080 10GB at Q8_0, at roughly 104.5 tokens per second and from $0.17 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.