CanItRun Logocanitrun.

Bonsai 2 4B

Bonsai 2 4B doesn't use the standard quantization ladder; it ships as fixed builds needing 1.4 GB at Ternary (PTQ1_0) or 1.6 GB at Ternary (PQ2_0) or 1.7 GB at Ternary (Q2_0). 117 GPUs we track can run at least one build fully in VRAM at 8k context.

117 GPUs run this natively · 1 with CPU offload

PrismML4B params256k contextApache 2.0Commercial use ok
Ternary (PQ2_0) total
1.6 GB
at 8k context
Smallest GPU
4 GB
Intel Arc A310 4GB, at Ternary (PQ2_0)
KV cache, full context
8.6 GB
8 of 32 layers cache
Inputs
Text · Image
Apache 2.0, released October 2026
Knowledge retention
90.9%
PrismML's own suite vs Qwen3.5-4B
Coding retention
100.7%
HumanEval+ 85.37 vs 83.54

Bonsai 2 4B is a 4B parameter dense model developed by PrismML. Released October 15, 2026 by PrismML as the middle size of the Bonsai 2 small-model release: a ternary requantization of Alibaba's Qwen3.5-4B, same architecture, not a retrain. Apache 2.0, text plus image input, 262,144-token context inherited from the base model. Three packings of the same weights: PTQ1_0 (0.95 GB), PQ2_0 (1.15 GB, PrismML's default) and Q2_0 (1.21 GB, readable by upstream llama.cpp b11555 or newer); like the 9B it also has an MTP variant of the PQ2_0 file.

To run Bonsai 2 4B locally: At 1.15 GB of weights (1.6 GB in total at 8k context) it runs on nearly any recent GPU, a thin-and-light laptop, or a phone. PrismML publishes no throughput figure for this size. As with the other two, the 262,144-token window is dominated by KV cache (8.6 GB), not weights. PTQ1_0, PQ2_0 and the MTP variant need PrismML's llama.cpp fork; Q2_0 runs on stock llama.cpp b11555 or newer.

PrismML's own table puts the 4B ahead of or level with full-precision Qwen3.5-4B on coding (HumanEval+ 85.37 vs 83.54), instruction following (IFEval 88.91 vs 86.69) and math (MATH500 97.6 vs 99.0, GSM8K level at 94.54), while knowledge retention is 90.9% overall (GPQA Diamond 63.84 vs 78.08). The weak spot is agentic work: Tau2Bench falls to 61.87 vs 81.65 (75.8% retained), the 4B's weakest score by a wide margin. Vendor-reported, not yet independently reproduced.

The cache is several times the weights at long context

Ternary quantization shrinks the weights and nothing else: the KV cache stays 16-bit. For a 1.15 GB model that turns the full context window into the dominant cost.

01020304032k128k192k256k8.6 GBBonsai 2 4B34.4 GBall-attention 4B
Bonsai 2 4B: 8 of 32 layers cacheThe same model if all 32 layers used full attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Only 8 of the 32 layers keep a KV cache, so the full 262,144-token window costs 8.6 GB of cache, against 34.4 GB if every layer cached. That is about seven and a half times the 1.15 GB of PQ2_0 weights: a full-window session needs 10.9 GB in total, versus 1.6 GB at 8k context. The 4B and the 9B share the same cache cost, since both use 8 caching layers with 4 KV heads of 256 dimensions, which is one more reason to pick between them on quality and not on context.

Available builds

Bonsai 2 4B skips the standard FP32–Q2_K quantization ladder entirely; it ships as 3 fixed, downloadable builds instead. Sizes below assume 8k of context.

BuildBits/weightWeightsKV cacheTotalQuality vs FP16
Ternary (PTQ1_0)1.810.9 GB0.27 GB1.4 GBN/A
Ternary (PQ2_0)2.181.1 GB0.27 GB1.6 GBN/A
Ternary (Q2_0)2.311.2 GB0.27 GB1.7 GBN/A

*Vendor-reported on the vendor's own benchmark suite, not yet independently verified by a third-party leaderboard. KV cache figures assume 8k context at FP16.

Benchmarks

GPQA Diamond
63.8
IFEval
88.9

Estimated, not measured: derived from the base model's real score times the best build's vendor-reported quality retention (see the chart above), not independently verified, and not a leaderboard result for Bonsai 2 4B itself.

PrismML's own benchmark table: what ternary weights cost a 4B

PrismML's launch table runs the ternary 4B and full-precision Qwen3.5-4B through the same harness. A sample of eight of its 19 benchmarks, spanning every category it reports:

BFCLv3 (tool calling)
Bonsai 2 4B
70.2
Qwen 3.5 4B base
71.5
Tau2Bench (agentic)
Bonsai 2 4B
61.9
Qwen 3.5 4B base
81.7
MATH500
Bonsai 2 4B
97.6
Qwen 3.5 4B base
99.0
IFEval
Bonsai 2 4B
88.9
Qwen 3.5 4B base
86.7
HumanEval+
Bonsai 2 4B
85.4
Qwen 3.5 4B base
83.5
MMLU-Redux
Bonsai 2 4B
82.0
Qwen 3.5 4B base
88.6
GPQA Diamond
Bonsai 2 4B
63.8
Qwen 3.5 4B base
78.1
Omni_doc (vision)
Bonsai 2 4B
71.2
Qwen 3.5 4B base
86.5

PrismML, Bonsai 2 family announcement (prismml.com/news/bonsai-2-family), 15 October 2026. PrismML's own harness, run on the ternary build and on full-precision Qwen3.5-4B side by side; not yet listed on any independent leaderboard or independently reproduced.

Math, coding and instruction following hold at 99 to 103% of the base model on the rows shown. The agentic row is the outlier: Tau2Bench keeps only 75.8% (61.87 vs 81.65), the largest drop in the 4B's table, while single-call tool use (BFCLv3) keeps 98.1%. That gap suggests the 4B handles one well-formed tool call fine but loses the thread over a multi-turn task. GPQA Diamond keeps 81.8% and document-style vision (Omni_doc) 82.3%. PrismML's own harness, not independently reproduced.

GPUs that run Bonsai 2 4B natively (117)

Show 112 more
Plus 1 GPU that runs it with CPU offload (slower)

No GPU? Rent an RTX 3080 10GB from $0.17/hr ↗

Prices as of Oct 13, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

Third-party ternary requantization of Qwen3.5-4B by PrismML, same architecture, not a retrain, and part of the Bonsai 2 small-model release of 15 October 2026 (2B, 4B, 9B). Doesn't use the standard FP32-Q2_K ladder shown for other models; it ships as three fixed packings of the same ternary weights (see the quantization table on this page): PTQ1_0 and PQ2_0 need PrismML's own llama.cpp fork, while Q2_0 runs on upstream llama.cpp b11555 or newer. File sizes are the language model only; the Q8_0 vision projector adds 0.37 GB. GPQA Diamond and IFEval are PrismML's own measurements of this build in its in-house harness, not independently verified.

Hugging Face ↗Released 2026-10-15

Compare Bonsai 2 4B with other models

How to run Bonsai 2 4B locally

PrismML's llama.cpp fork (PQ2_0 packing, PrismML's own sampling)

./llama-server -m Ternary-Bonsai-2-4B-PQ2_0.gguf \
  --mmproj Ternary-Bonsai-2-4B-mmproj-Q8_0.gguf \
  -ngl 99 -fa on -c 32768 --jinja --port 8080 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 \
  --presence-penalty 1.0 --repeat-penalty 1.0 --repeat-last-n 64

llama.cpp

# Build PrismML's llama.cpp fork (needed for PTQ1_0 and PQ2_0; Q2_0 also runs on upstream b11555+)
git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON   # drop the flag on macOS (Metal is default)
cmake --build build -j

# Download the weights and the Q8_0 vision projector (not the BF16 one, it can crash llama-server under concurrent image requests)
hf download prism-ml/Ternary-Bonsai-2-4B-gguf Ternary-Bonsai-2-4B-PQ2_0.gguf Ternary-Bonsai-2-4B-mmproj-Q8_0.gguf --local-dir .

# Optional: MTP speculative decoding (load this file INSTEAD of the plain PQ2_0, 1.27 GB)
./llama-server -m Ternary-Bonsai-2-4B-PQ2_0-MTP-Q8_0.gguf -ngl 99 -fa on \
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0

LM Studio: Stock LM Studio bundles upstream llama.cpp, so only the Q2_0 file (1.21 GB) is a candidate and it needs a runtime based on llama.cpp b11555 or newer; PTQ1_0 and PQ2_0 will be rejected. On Apple Silicon the MLX build (prism-ml/Ternary-Bonsai-2-4B-mlx-2bit, 2.05 GB on disk) is the other supported route.

Why this quantization? PQ2_0 is the packing PrismML's own run commands use: 1.15 GB of weights and 1.6 GB in total at 8k context, against 0.95 GB (1.4 GB total) for the denser PTQ1_0 file that needs the same fork. Q2_0 (1.21 GB, 1.7 GB total) is the one to pick only if you need stock upstream llama.cpp. All three hold the same ternary weights, so the choice is file size against runtime support. The weights are tiny next to the cache: the full 262,144-token window adds 8.6 GB of KV cache, so set -c explicitly instead of -c 0.

Who is Bonsai 2 4B for?

People who want the smallest model that still does vision, reasoning and tool calling acceptably well: phones, thin laptops, single-board computers, or a background service on a machine that is already busy.

Best for

  • On-device assistants where a 9B's 2.4 GB of weights is too much
  • Instruction-following and coding help, where PrismML's table shows it level with Qwen3.5-4B
  • Screenshot, chart and document questions through the Q8_0 vision projector
  • Short-context tool calling with --jinja

Not ideal for

  • Multi-step agent loops: Tau2Bench retention is only 75.8% in PrismML's own table
  • Knowledge and reasoning questions (GPQA Diamond 63.84 vs 78.08 for full-precision Qwen3.5-4B)
  • Stock Ollama or LM Studio users who do not want PrismML's llama.cpp fork (only the Q2_0 file runs upstream)
  • Anyone who needs benchmark claims independently verified before adopting a model; released the same day this page was written

Where does the 4B fit between the 2B and the 9B?

Pick the 4B when even the 9B's 2.42 GB of PQ2_0 weights is too much, or when speed matters more than depth: it needs 1.15 GB of weights and 1.6 GB in total at 8k context, 1.27 GB less weight than the 9B. Otherwise the 9B is the better trade. PrismML's own numbers show the 4B giving up real ground on GPQA Diamond (63.84 vs the 9B's 69.9) and multi-step agent work (Tau2Bench 61.87 vs 73.02), and both sizes need the same KV cache at long context, so the extra download buys quality at almost no extra cache cost.

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Bonsai 2 4B?
Bonsai 2 4B doesn't use the standard quantization ladder; it ships as 3 fixed builds: Ternary (PTQ1_0) (1.4 GB), Ternary (PQ2_0) (1.6 GB), and Ternary (Q2_0) (1.7 GB). These figures assume 8k of context; VRAM scales linearly with context length due to the KV cache.
How many parameters does Bonsai 2 4B have?
Bonsai 2 4B has 4 billion parameters.
What quantization levels does Bonsai 2 4B come in?
Bonsai 2 4B skips the usual FP16-to-Q2_K ladder entirely. It's only available as: Ternary (PTQ1_0) at 1.81 bits/weight (1.4 GB total); Ternary (PQ2_0) at 2.18 bits/weight (1.6 GB total); Ternary (Q2_0) at 2.31 bits/weight (1.7 GB total).
Can Bonsai 2 4B run on a small GPU?
Yes. The Ternary (PTQ1_0) build needs only 1.4 GB, which fits on most 8 GB+ GPUs, entry-level Apple Silicon, and even high-end phones per the vendor's own claims.
What GPU do I need to run Bonsai 2 4B locally?
Every build is small enough for almost any GPU released in the last several years. The Ternary (Q2_0) build (1.7 GB) needs a bit more headroom for better quality, but all of them fit comfortably on an 8 GB card or larger.
Can I run Bonsai 2 4B in the cloud?
Yes. The cheapest RunPod setup we found that holds Bonsai 2 4B entirely in GPU memory at 8k context is 1× RTX 3080 10GB at Ternary (Q2_0), at roughly 242.1 tokens per second and from $0.17 per hour for the whole pod. Prices as of Oct 13, 2026; RunPod rates change, so check their pricing page for the current figure.
Which Bonsai 2 4B file should I download?
The PQ2_0 file (1.15 GB) is the default in PrismML's docs and run commands and needs PrismML's llama.cpp fork, as does the smaller PTQ1_0 (0.95 GB). Q2_0 (1.21 GB) is the one stock upstream llama.cpp b11555 or newer can read. All three hold the same ternary weights. The Q8_0 vision projector adds 0.37 GB.
Is the 4B good enough for agent workflows?
For single tool calls, probably: PrismML's BFCLv3 result keeps 98.1% of the base model. For multi-turn agent tasks it is the weakest of the small sizes relative to its base, with Tau2Bench retention of 75.8%, so test your own workflow before depending on it, or step up to the 9B.