Step 3.5 Flash
Step 3.5 Flash needs roughly 135.9 GB VRAM at Q4_K_M quantization (442.6 GB at FP16). 20 GPUs we track can run it fully in VRAM at 8k context.
20 GPUs run this natively · 2 with CPU offload
Step 3.5 Flash is a Mixture of Experts (MoE) large language model with 196.81B total parameters but only 11B active per token developed by StepFun. Released in February 2026, it is a text-only model with a 256K context window, released under the Apache 2.0 license, allowing commercial use. Coding- and reasoning-focused MoE with 288 routed experts, 8 selected per token plus a shared expert, a 3:1 sliding/full-attention pattern, and three multi-token-prediction heads. Scores are StepFun-reported: MMLU-Pro 84.4, GPQA-Diamond 83.5, and LiveCodeBench v6 86.4. An official Q4_K_S GGUF is available, but it still requires server-class memory.
To run Step 3.5 Flash locally, you need approximately 135.9 GB of VRAM at Q4_K_M quantization with 8k context. 20 of the GPUs we track can run it fully in VRAM, with a further 2 able to offload to system RAM. At Q4_K_M it requires 135.9 GB — more than any single consumer GPU. A multi-GPU server or 80 GB datacenter GPU is required. At Q8_K_M (236.0 GB), you get near-FP16 quality while still fitting on large server or multi-GPU setups. FP16 requires 442.5 GB, limiting it to datacenter-class hardware with 80 GB+ VRAM. As a MoE model, inference speed depends on active parameters (11B) rather than total size, so once it fits in VRAM it runs noticeably faster than dense models of the same parameter count.
Its MMLU-Pro score of 84.4 places it among the strongest open-weight models available. Its LiveCodeBench score of 86.4 makes it a solid choice for code generation and completion. The license allows commercial use.
VRAM at each quantization
Figures below assume 8k context; KV cache grows linearly as context length increases.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 787.2 GB | 1.51 GB | 883.4 GB |
| BF16 | 393.6 GB | 1.51 GB | 442.6 GB |
| FP16 | 393.6 GB | 1.51 GB | 442.6 GB |
| Q8_0 | 209.2 GB | 1.51 GB | 236.0 GB |
| Q6_K | 161.6 GB | 1.51 GB | 182.7 GB |
| Q5_K_M | 140.1 GB | 1.51 GB | 158.6 GB |
| Q4_K_Mrec | 119.9 GB | 1.51 GB | 135.9 GB |
| Q3_K_M | 94.7 GB | 1.51 GB | 107.7 GB |
| Q2_K | 75.0 GB | 1.51 GB | 85.7 GB |
| NVFP4cuda | 98.4 GB | 1.51 GB | 111.9 GB |
KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.
Benchmarks
GPUs that run Step 3.5 Flash natively (20)
- NVIDIA RTX Pro 6000Q2_K · 56.4 t/s
- NVIDIA DGX Spark (128GB)NVFP4 · 8.9 t/s
- AMD Instinct MI300XQ5_K_M · 124.7 t/s
- AMD Strix Halo (128GB)Q3_K_M · 8.7 t/s
- AMD Strix Halo (96GB)Q2_K · 10.7 t/s
- Apple M5 Max (128GB)Q3_K_M · 25.7 t/s
- Apple M4 Ultra (384GB)Q8_0 · 21.6 t/s
- Apple M4 Ultra (192GB)Q6_K · 27.6 t/s
- Apple M4 Max (128GB)Q3_K_M · 22.8 t/s
- Apple M4 Max (96GB)Q2_K · 28.2 t/s
- Apple M3 Ultra (512GB)BF16 · 8.8 t/s
- Apple M3 Ultra (256GB)Q8_0 · 16.2 t/s
- Apple M3 Ultra (96GB)Q2_K · 42.3 t/s
- Apple M3 Max (128GB)Q3_K_M · 16.7 t/s
- Apple M3 Max (96GB)Q2_K · 20.7 t/s
- Apple M2 Ultra (384GB)Q8_0 · 15.8 t/s
- Apple M2 Ultra (192GB)Q6_K · 20.2 t/s
- Apple M2 Max (96GB)Q2_K · 20.7 t/s
- Apple M1 Ultra (128GB)Q3_K_M · 33.4 t/s
- Intel Data Center GPU Max 1550Q3_K_M · 111.2 t/s
Plus 2 GPUs that run it with CPU offload (slower)
- NVIDIA H100 80GBQ2_K · 90.4 t/s
- NVIDIA A100 80GBQ2_K · 64.1 t/s
Notes
Coding- and reasoning-focused MoE with 288 routed experts, 8 selected per token plus a shared expert, a 3:1 sliding/full-attention pattern, and three multi-token-prediction heads. Scores are StepFun-reported: MMLU-Pro 84.4, GPQA-Diamond 83.5, and LiveCodeBench v6 86.4. An official Q4_K_S GGUF is available, but it still requires server-class memory.
How to run Step 3.5 Flash locally
Q4_K_M needs 135.9 GB — needs multiple datacenter-class GPUs (80 GB+ each).
vLLM
vllm serve stepfun-ai/Step-3.5-Flashllama.cpp
./llama-cli -m Step-3.5-Flash-Q4_K_S.gguf -c 32768 -ngl 99LM Studio: Use the official Q4_K_S GGUF only on systems with roughly 128 GB or more combined VRAM/unified memory after allowing for KV cache and runtime overhead.
Why this quantization? StepFun publishes an official Q4_K_S conversion, making Q4 the best evidence-backed quality/size target. At almost 197B total parameters it is still a server or large unified-memory model; dropping to Q2 saves memory but sacrifices too much of its coding and reasoning quality.
Who is Step 3.5 Flash for?
Developers with high-memory workstations or multi-GPU servers who want a strong open coding/reasoning model and can accommodate a roughly 120 GB-class Q4 weight file.
Best for
- Agentic software engineering
- Algorithmic coding with LiveCodeBench-v6 strength
- Long reasoning traces and tool calls
Not ideal for
- 24-48 GB single-GPU systems
- Vision tasks
- Latency-sensitive autocomplete
Frequently asked questions
- What are the VRAM requirements for Step 3.5 Flash?
- Step 3.5 Flash requires approximately 135.9 GB of VRAM at Q4_K_M quantization, 236.0 GB at Q8, and 442.5 GB at FP16. These numbers assume 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Step 3.5 Flash have?
- Step 3.5 Flash has 196.81 billion total parameters, but only 11 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- Is Step 3.5 Flash good for coding?
- Yes. Step 3.5 Flash scores 86.4 on LiveCodeBench, demonstrating strong code generation and completion capabilities.
- Can Step 3.5 Flash run on a 16 GB GPU?
- No. At Q4_K_M, Step 3.5 Flash needs 135.9 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Step 3.5 Flash run on a 24 GB GPU?
- No. Even at Q4_K_M, Step 3.5 Flash needs 135.9 GB. Consider a multi-GPU server with 80 GB+ total VRAM.
- What is the smallest quantization for Step 3.5 Flash that fits in 24 GB of VRAM?
- Step 3.5 Flash cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 85.7 GB at Q2_K.
- What GPU do I need to run Step 3.5 Flash locally?
- You need a multi-GPU server. At Q4_K_M, Step 3.5 Flash needs 135.9 GB VRAM, more than any single consumer GPU. Consider 2–4× H100 or A100 GPUs.