CanItRun Logocanitrun.

Flint 40B (MoE)

Flint 40B (MoE) needs roughly 27.4 GB VRAM at Q4_K_M quantization (89.5 GB at FP16). 70 GPUs we track can run it fully in VRAM at 8k context.

70 GPUs run this natively · 33 with CPU offload

Recursive39.8B params2.5B active (MoE)256k contextApache 2.0Commercial use ok
Q4_K_M total
27.4 GB
at 8k context
Smallest GPU
32 GB
NVIDIA RTX 5090, at Q4_K_M
KV cache, full context
7.6 GB
14 of 49 layers grow with context, 35 capped at 513 tokens
Inputs
Text
Apache 2.0, released October 2026
Active experts
6 of 256 + 1 shared
~2.5B active per token
Attention pattern
35 sliding : 14 full
513-token window

Flint 40B (MoE) is a Mixture of Experts (MoE) model with 39.8B total parameters but only 2.5B active per token developed by Recursive. Released on 21 October 2026 by Recursive under the Apache 2.0 license, a mixture-of-experts model trained from scratch (the card names no base model) for coding agents. It has 39.8B parameters in total and about 2.5B active per token: 49 layers, the first two dense and the other 47 each routing every token to 6 of 256 experts plus an always-on shared expert. Its attention stack is the unusual part: 14 layers look at the whole context and the other 35 use a 513-token sliding window, so most of the KV cache is capped. Context is 262,144 tokens, shared between prompt and output, and the launch post recommends staying within 256k. It is text only, reasons before answering, and supports multi-turn tool calls.

To run Flint 40B (MoE) locally: The weights are the cost, not the context. Q4_K_M comes to 24.2 GB by this site's estimate, in line with the published files (24.2 GB from lmstudio-community, 25.4 GB from bartowski), and Recursive's own NVFP4 GGUF is 22.9 GB. The full 262,144-token window adds only 7.6 GB of KV cache, against 26.3 GB if all 49 layers cached the whole context. A 24 GB card such as the RTX 3090 or RTX 4090 cannot hold Q4_K_M with any working context and drops to a 3-bit build (19.1 GB of weights), which fits at a short context but spills to system RAM at the full window (29.9 GB in total). A 32 GB RTX 5090 holds Q4_K_M at a short context and a 3-bit build at the full window, and a 64 GB Mac holds even Q8_0 at the full window. Recursive reports more than 125 tokens per second on an M4 Pro MacBook Pro and more than 150 on a 64GB NVIDIA RTX Spark at short contexts, its own figures for its own engine. The card's serving kit targets a single B300 or H200, so the NVFP4 and int4 GGUFs are the local routes. As a MoE model, inference speed depends on active parameters (2.5B) rather than total size.

Recursive's card reports Terminal-Bench 2.1 at 54.9 and SWE-bench Verified at 74.2, each an average pass@1 over at least five attempts (Terminus 2 and mini-swe-agent harnesses respectively), and counts Terminal-Bench solutions found through web retrieval as failures. It also lists 89.7 on SWE-bench Pro v2, a different suite from the Scale AI SWE-bench Pro leaderboard this site tracks, so that figure is quoted here but not entered in the benchmark table. The launch post claims Flint leads MoE models under 50B total parameters on agentic coding and rivals dense models of 25 to 40B at more than five times the speed, but it names no comparison models and gives no scores in text. These are Recursive's own runs, and no independent write-up had appeared when this page was written. The post also warns that Flint may need more guidance than the larger closed coding models, so split complex tasks into subtasks.

What a full 262,144-token context actually costs

Long context is where a local coding agent usually runs out of memory: every token an agent reads adds to the KV cache. Flint keeps that bill small by letting 35 of its 49 attention layers look back only 513 tokens, however long the session gets. Only the other 14 layers keep a cache that grows with context.

081624328k64k128k256k513-token window7.6 GBFlint26.3 GBno window
Flint 40B (MoE): 14 of 49 layers grow with contextThe same 40B if all 49 layers used full, uncapped attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Filling the whole 262,144-token window costs 7.6 GB of KV cache instead of 26.3 GB. Add 24.2 GB of Q4_K_M weights and this site's full-window total is 35.6 GB; the same model with every layer caching the whole context would need 56.6 GB. The weights, not the context, set the hardware bill.

VRAM at each quantization

Calculated at 8k context. Because only part of this model's stack keeps a growing KV cache, longer sessions cost far less extra VRAM than a full-attention model of the same size.

QuantWeightsKV cacheTotal
FP32159.2 GB0.27 GB178.6 GB
BF1679.6 GB0.27 GB89.5 GB
FP1679.6 GB0.27 GB89.5 GB
Q8_042.3 GB0.27 GB47.7 GB
Q6_K32.7 GB0.27 GB36.9 GB
Q5_K_M28.3 GB0.27 GB32.0 GB
Q4_K_Mrec24.2 GB0.27 GB27.4 GB
Q3_K_M19.1 GB0.27 GB21.8 GB
Q2_K15.2 GB0.27 GB17.3 GB
NVFP4cuda22.9 GB0.27 GB25.9 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Quantization × context: where it fits

The table above holds context fixed at 8k. Add context as a second axis and the picture changes little for this model, because most layers stop growing after 513 tokens. Pick the row you can download, then the column you actually need.

Total VRAM needed by quantization and context length, with the GPU class each figure requires.
Quant8k ctx32k ctx128k ctx256k ctx
Q8_047.7 GB48 GB card48.5 GB80 GB card51.6 GB80 GB card55.8 GB80 GB card
Q6_K36.9 GB48 GB card37.7 GB48 GB card40.9 GB48 GB card45.1 GB48 GB card
Q5_K_M32.0 GB48 GB card32.8 GB48 GB card36.0 GB48 GB card40.2 GB48 GB card
Q4_K_Mrec27.4 GB32 GB card28.2 GB32 GB card31.4 GB32 GB card35.6 GB48 GB card
Q3_K_M21.8 GB24 GB card22.5 GB24 GB card25.7 GB32 GB card29.9 GB32 GB card
Each cell shows the smallest card that holds it:Consumer GPUWorkstation GPUDatacenter GPU

Benchmarks

SWE-bench Verified
74.2
Terminal-Bench 2.1
54.9

GPUs that run Flint 40B (MoE) natively (70)

Show 65 more
Plus 33 GPUs that run it with CPU offload (slower)

No GPU? Rent an RTX A6000 from $0.33/hr ↗

Prices as of Oct 17, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

Released 21 October 2026 by Recursive, trained from scratch (the card names no base model) for coding agents: 39.8B parameters with about 2.5B active per token across 256 routed experts, 6 chosen per token plus a shared expert. Only 14 of its 49 attention layers look at the whole context; the other 35 use a 513-token sliding window, so the KV cache stays small as context grows. Recursive's card reports Terminal-Bench 2.1 54.9 and SWE-bench Verified 74.2 (average pass@1 over at least five attempts), plus 89.7 on SWE-bench Pro v2, a different suite from the Scale AI leaderboard this site tracks, so it is not entered in the SWE-bench Pro field. The launch post claims it leads MoE models under 50B total parameters on agentic coding and reports more than 125 tokens per second on an M4 Pro MacBook Pro and more than 150 on a 64GB NVIDIA RTX Spark at short contexts; these are Recursive's own measurements, and the post names no comparison models or scores. The post also says Flint may need more guidance than the larger closed coding models, so break complex tasks into subtasks. The BF16 checkpoint is 79.6GB; Recursive also publishes NVFP4 (22.9GB) and int4 (26.0GB Q4_1) GGUFs, and the Ollama tag flint:40b is 23GB. Apache 2.0, text only.

Hugging Face ↗Ollama ↗Released 2026-10-21

Compare Flint 40B (MoE) with other models

How to run Flint 40B (MoE) locally

816244880160320

Q4_K_M needs 27.4 GB: needs a workstation or datacenter GPU (48–80 GB).

llama.cpp (a recent build with Flint support), or Ollama, LM Studio, vLLM and SGLang

./llama-server \
  -hf lmstudio-community/flint-GGUF:Q4_K_M \
  -c 32768 -ngl 99 -fa on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0

Ollama

ollama run flint:40b

llama.cpp

# Recursive's own NVIDIA build (Blackwell): 22.9 GB NVFP4 file at the full 262,144-token window
hf download RecursiveSI/flint-nvfp4-gguf flint-nvfp4.gguf --local-dir flint-nvfp4-gguf
llama-server -m flint-nvfp4-gguf/flint-nvfp4.gguf \
  --alias flint -ngl all -c 262144 -np 1 \
  -fa on -ctk f16 -ctv f16 --jinja \
  --temp 1 --top-p 0.95 --top-k 20 --min-p 0

# Leave --swa-full unset so the 35 sliding-window layers keep their compact cache

LM Studio: Search for 'flint' and take the Q4_K_M build from lmstudio-community (24.2 GB), or Q6_K and Q8_0 if you have the memory. The model reasons before it answers, so expect a think block, and keep the reasoning in the conversation history between turns (the chat template replays it). Temperature 1.0, top-p 0.95 and top-k 20 are Recursive's settings. Because only 14 of 49 layers grow their cache, raising the context slider costs far less memory than on a conventional model; the weights are the budget.

Why this quantization? Q4_K_M puts the weights at 24.2 GB by this site's estimate, close to the published files (24.2 GB from lmstudio-community, 25.4 GB from bartowski). Context is the cheap part: the full 262,144-token window adds only 7.6 GB of KV cache, because 35 of the 49 layers cap their cache at a 513-token window. That makes the memory plan simple: a 32 GB card holds Q4_K_M at a short context, a 64 GB Mac holds it at the full window, and a 24 GB card has to step down to a 3-bit build (19.1 GB of weights). On Blackwell GPUs, Recursive's own 22.9 GB NVFP4 GGUF is the build the company tuned for, and Recursive says its NVFP4 checkpoint scores within 0.7 points of the 16-bit model across its benchmarks.

Who is Flint 40B (MoE) for?

Developers who want a local coding agent that decodes fast on a single 32 GB card or a 48 to 64 GB Apple Silicon machine. Only about 2.5B parameters are active per token, so decode speed is closer to a 3B model's than a 40B model's, while all 39.8B parameters still have to be stored. Recursive pitches it for investigating codebases, fixing bugs, implementing features and writing tests through an OpenAI-compatible tool-calling server.

Best for

  • Local coding agents (pi, OpenCode and similar) on a 32 GB GPU or a 48 to 64 GB Mac, where the KV cache stays small even at long context
  • Repository-scale tasks that need a long window without a large memory bill, since only 14 of 49 layers grow their cache
  • Terminal and tool-use work through llama-server's OpenAI-compatible API with --jinja enabled

Not ideal for

  • 16 GB GPUs, where even the 2-bit build (about 17 GB in total) spills to system RAM
  • Anyone who needs vision or audio input, it is text only
  • Anyone who needs the benchmark claims independently verified before adopting a model, it was released the day before this page was written and every score is Recursive's own
  • Open-ended tasks with no scaffolding, Recursive's own post says it may need more guidance than the larger closed coding models

Which card do you need?

Choose by the weights, not the context. Q4_K_M is 24.2 GB of weights and the full 262,144-token window adds 7.6 GB, for 35.6 GB in total. A 32 GB RTX 5090 runs Q4_K_M at a short context but spills to system RAM at the full window, where a 3-bit build (19.1 GB of weights, 29.9 GB in total) still fits. A 24 GB card such as the RTX 3090 or RTX 4090 is a 3-bit card even at a short context, and spills at the full window. A 64 GB Mac holds Q4_K_M at the full window, and even Q8_0. On Blackwell cards, Recursive's own 22.9 GB NVFP4 file is the build the company tuned for. A 16 GB card spills at every quantization. Every benchmark on this page is Recursive's own and the model was a day old when this page was written, so test it on your own repository before relying on it.

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Flint 40B (MoE)?
Flint 40B (MoE) requires approximately 27.4 GB of VRAM at Q4_K_M quantization, 47.7 GB at Q8, and 89.5 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Flint 40B (MoE) have?
Flint 40B (MoE) has 39.8 billion total parameters, but only 2.5 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
Can Flint 40B (MoE) run on a 16 GB GPU?
No. At Q4_K_M, Flint 40B (MoE) needs 27.4 GB of VRAM, more than 16 GB. You will need a 32 GB GPU like the RTX 5090.
Can Flint 40B (MoE) run on a 24 GB GPU?
No. Even at Q4_K_M, Flint 40B (MoE) needs 27.4 GB. Consider a 32 GB card like the RTX 5090.
What is the smallest quantization for Flint 40B (MoE) that fits in 24 GB of VRAM?
At Q3_K_M, Flint 40B (MoE) needs 21.8 GB, the highest-quality quantization that fits in 24 GB of VRAM.
What GPU do I need to run Flint 40B (MoE) locally?
A 32 GB GPU is the minimum. At Q4_K_M, Flint 40B (MoE) needs 27.4 GB VRAM. Good option: RTX 5090 (32 GB).
Can I run Flint 40B (MoE) in the cloud?
Yes. The cheapest RunPod setup we found that holds Flint 40B (MoE) entirely in GPU memory at 8k context is 1× RTX A6000 at Q6_K, at roughly 134.1 tokens per second and from $0.33 per hour for the whole pod. Prices as of Oct 17, 2026; RunPod rates change, so check their pricing page for the current figure.
How much VRAM does Flint need?
The Q4_K_M weights are 24.2 GB by this site's estimate, in line with the published files (24.2 GB from lmstudio-community, 25.4 GB from bartowski), and Recursive's own NVFP4 GGUF is 22.9 GB. The full 262,144-token window adds 7.6 GB of KV cache, for 35.6 GB in total at Q4_K_M. A 24 GB card runs a 3-bit build (19.1 GB of weights) at a short context but not the full window, which comes to 29.9 GB in total; a 32 GB card holds Q4_K_M at a short context and the 3-bit build at the full window.
Why does Flint's long context cost so little memory?
Because 35 of its 49 attention layers use a 513-token sliding window, their cache stops growing after 513 tokens. Only the other 14 layers cache the whole context, which holds the full 262,144-token window to 7.6 GB of KV cache; if all 49 layers cached everything it would be 26.3 GB. Recursive's own llama.cpp instructions say to leave --swa-full unset so the compact sliding-window cache stays on.
Can Flint run on a 16 GB GPU?
Not entirely in VRAM. Even the 2-bit build is larger than a 16 GB card can use once a working context is added, so part of the model runs from system RAM. Only about 2.5B of its 39.8B parameters are active per token, which softens the penalty compared with a dense model of the same size, but expect it to run slower than on a card that holds the whole model.
Are Flint's benchmark scores independent?
No. Recursive's card reports Terminal-Bench 2.1 at 54.9 and SWE-bench Verified at 74.2, averaged over at least five attempts, and its launch post makes comparative claims without naming the other models or giving scores in text. The card's 89.7 on SWE-bench Pro v2 is a different suite from the Scale AI SWE-bench Pro leaderboard this site tracks, so it is not entered in the benchmark table. The model was a day old when this page was written and no independent write-up had appeared.