CanItRun Logocanitrun.

Bonsai 2 2B

Bonsai 2 2B doesn't use the standard quantization ladder; it ships as fixed builds needing 0.6 GB at Ternary (PTQ1_0) or 0.7 GB at Ternary (PQ2_0) or 0.7 GB at Ternary (Q2_0). 117 GPUs we track can run at least one build fully in VRAM at 8k context.

117 GPUs run this natively · 1 with CPU offload

PrismML2B params256k contextApache 2.0Commercial use ok
Ternary (PQ2_0) total
0.7 GB
at 8k context
Smallest GPU
4 GB
Intel Arc A310 4GB, at Ternary (PQ2_0)
KV cache, full context
3.2 GB
6 of 24 layers cache
Inputs
Text · Image
Apache 2.0, released October 2026
Knowledge retention
82.9%
PrismML's own suite vs Qwen3.5-2B
Instruction retention
121.5%
base model scores low in this harness

Bonsai 2 2B is a 2B parameter dense model developed by PrismML. Released October 15, 2026 by PrismML as the smallest of the Bonsai 2 family: a ternary requantization of Alibaba's Qwen3.5-2B, same architecture, not a retrain. Apache 2.0, text plus image input, 262,144-token context inherited from the base model. Three packings of the same weights: PTQ1_0 (0.43 GB, the 'as small as 0.43GB' headline figure), PQ2_0 (0.52 GB, PrismML's default) and Q2_0 (0.55 GB). Unlike the 4B and 9B it has no MTP variant or drafter.

To run Bonsai 2 2B locally: At 0.52 GB of weights (0.7 GB in total at 8k context) it fits almost anything, including wearables and phones; PrismML reports up to 276 tok/s on an RTX 4090 or M5 Pro and up to 120 tok/s on an iPhone 18 Pro Max, its own figures. Even here the full 262,144-token window needs 3.2 GB of KV cache, several times the weights. PTQ1_0 and PQ2_0 need PrismML's llama.cpp fork; Q2_0 is the file PrismML's model card points Ollama and Docker Model Runner at.

PrismML's own table shows the 2B level with or ahead of full-precision Qwen3.5-2B on instruction following (IFEval 85.77 vs 75.79), coding (HumanEval+ 50.0 vs 45.12) and GSM8K (80.74 vs 65.96), though gains that large over a base model usually say more about that base model's weak showing in this harness than about quantization improving anything. Knowledge is where it pays: GPQA Diamond 39.6 vs 58.48 (67.7% retained) and MMLU-Redux 67.6 vs 79.39 (85.1%). All vendor-reported, not yet independently reproduced.

Even a 0.5 GB model needs gigabytes at long context

Ternary quantization shrinks the weights and nothing else: the KV cache stays 16-bit. For a half-gigabyte model, the full context window is the whole story.

047111432k128k192k256k3.2 GBBonsai 2 2B12.9 GBall-attention 2B
Bonsai 2 2B: 6 of 24 layers cacheThe same model if all 24 layers used full attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Only 6 of the 24 layers keep a KV cache, so the full 262,144-token window costs 3.2 GB of cache, against 12.9 GB if every layer cached. That is about six times the 0.52 GB of PQ2_0 weights: a full-window session needs 4.2 GB in total, versus 0.7 GB at 8k context. For the short-context jobs a 2B suits, the weights are the whole bill; for long documents, plan around the cache.

Available builds

Bonsai 2 2B doesn't offer the standard quantization ladder; instead there are 3 fixed builds to choose from. Sizes assume 8k context throughout.

BuildBits/weightWeightsKV cacheTotalQuality vs FP16
Ternary (PTQ1_0)1.830.4 GB0.10 GB0.6 GBN/A
Ternary (PQ2_0)2.20.5 GB0.10 GB0.7 GBN/A
Ternary (Q2_0)2.330.6 GB0.10 GB0.7 GBN/A

*These quality figures come from the vendor's own benchmarks and haven't been confirmed by an independent leaderboard yet. KV cache is shown at 8k context (FP16).

Benchmarks

GPQA Diamond
39.6
IFEval
85.8

Estimated, not measured: derived from the base model's real score times the best build's vendor-reported quality retention (see the chart above), not independently verified, and not a leaderboard result for Bonsai 2 2B itself.

PrismML's own benchmark table: what ternary weights cost a 2B

PrismML's launch table runs the ternary 2B and full-precision Qwen3.5-2B through the same harness. A sample of eight of its 19 benchmarks, spanning every category it reports:

BFCLv3 (tool calling)
Bonsai 2 2B
60.5
Qwen 3.5 2B base
62.1
MATH500
Bonsai 2 2B
82.4
Qwen 3.5 2B base
85.4
GSM8K
Bonsai 2 2B
80.7
Qwen 3.5 2B base
66.0
IFEval
Bonsai 2 2B
85.8
Qwen 3.5 2B base
75.8
HumanEval+
Bonsai 2 2B
50.0
Qwen 3.5 2B base
45.1
MMLU-Redux
Bonsai 2 2B
67.6
Qwen 3.5 2B base
79.4
GPQA Diamond
Bonsai 2 2B
39.6
Qwen 3.5 2B base
58.5
Omni_doc (vision)
Bonsai 2 2B
55.6
Qwen 3.5 2B base
71.8

PrismML, Bonsai 2 family announcement (prismml.com/news/bonsai-2-family), 15 October 2026. PrismML's own harness, run on the ternary build and on full-precision Qwen3.5-2B side by side; not yet listed on any independent leaderboard or independently reproduced.

Instruction following, coding and some math beat the base model outright (IFEval 85.77 vs 75.79, GSM8K 80.74 vs 65.96). Treat those gains with suspicion: a 2B base model that scores 65.96 on GSM8K in this harness is probably failing to follow the answer format, which a requantized build can happen to fix, and PrismML does not explain the effect. What is not in doubt is the other side of the table: GPQA Diamond keeps 67.7% (39.6 vs 58.48), MMLU-Redux 85.1%, and document-style vision (Omni_doc) 77.5%. At this size, ternary weights cost real knowledge.

GPUs that run Bonsai 2 2B natively (117)

Show 112 more
Plus 1 GPU that runs it with CPU offload (slower)

No GPU? Rent an RTX 3080 10GB from $0.17/hr ↗

Prices as of Oct 13, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

Third-party ternary requantization of Qwen3.5-2B by PrismML, same architecture, not a retrain, and part of the Bonsai 2 small-model release of 15 October 2026 (2B, 4B, 9B). Doesn't use the standard FP32-Q2_K ladder shown for other models; it ships as three fixed packings of the same ternary weights (see the quantization table on this page): PTQ1_0 and PQ2_0 need PrismML's own llama.cpp fork, while Q2_0 runs on upstream llama.cpp b11555 or newer. File sizes are the language model only; the Q8_0 vision projector adds 0.36 GB. GPQA Diamond and IFEval are PrismML's own measurements of this build in its in-house harness, not independently verified.

Hugging Face ↗Released 2026-10-15

Compare Bonsai 2 2B with other models

How to run Bonsai 2 2B locally

PrismML's llama.cpp fork (PQ2_0 packing, PrismML's own sampling)

./llama-server -m Ternary-Bonsai-2-2B-PQ2_0.gguf \
  --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf \
  -ngl 99 -fa on -c 32768 --jinja --port 8080 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 \
  --presence-penalty 1.0 --repeat-penalty 1.0 --repeat-last-n 64

llama.cpp

# Build PrismML's llama.cpp fork (needed for PTQ1_0 and PQ2_0; Q2_0 also runs on upstream b11555+)
git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON   # drop the flag on macOS (Metal is default)
cmake --build build -j

# Download the weights and the Q8_0 vision projector (not the BF16 one, it can crash llama-server under concurrent image requests)
hf download prism-ml/Ternary-Bonsai-2-2B-gguf Ternary-Bonsai-2-2B-PQ2_0.gguf Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf --local-dir .

# The Q2_0 file also loads on upstream llama.cpp b11555+ and, per PrismML's model card, through Ollama:
ollama run hf.co/prism-ml/Ternary-Bonsai-2-2B-gguf:Q2_0

LM Studio: Stock LM Studio bundles upstream llama.cpp, so only the Q2_0 file (0.55 GB) is a candidate and it needs a runtime based on llama.cpp b11555 or newer; PTQ1_0 and PQ2_0 will be rejected. On Apple Silicon the MLX build (prism-ml/Ternary-Bonsai-2-2B-mlx-2bit, 1.34 GB on disk) is the other supported route.

Why this quantization? PQ2_0 is the packing PrismML's own run commands use: 0.52 GB of weights and 0.7 GB in total at 8k context, against 0.43 GB (0.6 GB total) for the denser PTQ1_0 file that needs the same fork. Q2_0 (0.55 GB, 0.7 GB total) is the one to pick if you want stock upstream llama.cpp, Ollama or Docker Model Runner. All three hold the same ternary weights, so the choice is file size against runtime support. The 2B has no MTP variant or drafter. Set -c explicitly: the full 262,144-token window adds 3.2 GB of KV cache, many times the weights.

Who is Bonsai 2 2B for?

Anyone running a language model somewhere memory is genuinely scarce: wearables, phones, microcontroller-class Linux boards, or as a tiny always-resident helper next to something much bigger.

Best for

  • Fast on-device instruction following, formatting and extraction (PrismML reports up to 120 tok/s on an iPhone 18 Pro Max)
  • Simple tool calling and routing, where a tiny model decides which bigger one to call
  • Lightweight image captioning through the Q8_0 vision projector (the projector is 0.36 GB, almost as large as the weights)
  • A draft or triage model in front of a larger one

Not ideal for

  • Knowledge and reasoning: PrismML's own GPQA Diamond is 39.6 against 58.48 for full-precision Qwen3.5-2B
  • Document-heavy vision tasks (Omni_doc 55.64 vs 71.78)
  • Anything that needs claims independently verified; released the same day this page was written

What is the 2B actually for?

It is for places where nothing bigger fits: wearables, phones, or a helper that sits permanently in memory next to a larger model. It needs only 0.52 GB of weights and 0.7 GB in total at 8k context, and PrismML's own numbers show instruction following and simple tool calling holding up (IFEval 85.77, BFCLv3 60.53 vs 62.06 for the base). It is the wrong tool for knowledge or reasoning questions, where PrismML's own GPQA Diamond is 39.6. If a few hundred megabytes of extra weights are available, the 4B is a much stronger model for 1.15 GB.

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Bonsai 2 2B?
Bonsai 2 2B doesn't use the standard quantization ladder; it ships as 3 fixed builds: Ternary (PTQ1_0) (0.6 GB), Ternary (PQ2_0) (0.7 GB), and Ternary (Q2_0) (0.7 GB). These figures assume 8k of context; VRAM scales linearly with context length due to the KV cache.
How many parameters does Bonsai 2 2B have?
Bonsai 2 2B has 2 billion parameters.
What quantization levels does Bonsai 2 2B come in?
Bonsai 2 2B skips the usual FP16-to-Q2_K ladder entirely. It's only available as: Ternary (PTQ1_0) at 1.83 bits/weight (0.6 GB total); Ternary (PQ2_0) at 2.2 bits/weight (0.7 GB total); Ternary (Q2_0) at 2.33 bits/weight (0.7 GB total).
Can Bonsai 2 2B run on a small GPU?
Yes. The Ternary (PTQ1_0) build needs only 0.6 GB, which fits on most 8 GB+ GPUs, entry-level Apple Silicon, and even high-end phones per the vendor's own claims.
What GPU do I need to run Bonsai 2 2B locally?
Every build is small enough for almost any GPU released in the last several years. The Ternary (Q2_0) build (0.7 GB) needs a bit more headroom for better quality, but all of them fit comfortably on an 8 GB card or larger.
Can I run Bonsai 2 2B in the cloud?
Yes. The cheapest RunPod setup we found that holds Bonsai 2 2B entirely in GPU memory at 8k context is 1× RTX 3080 10GB at Ternary (Q2_0), at roughly 414.4 tokens per second and from $0.17 per hour for the whole pod. Prices as of Oct 13, 2026; RunPod rates change, so check their pricing page for the current figure.
Which Bonsai 2 2B file should I download?
The PQ2_0 file (0.52 GB) is the default in PrismML's docs and needs PrismML's llama.cpp fork, as does the smaller PTQ1_0 (0.43 GB, the 'as small as 0.43GB' headline figure). Q2_0 (0.55 GB) runs on stock upstream llama.cpp b11555 or newer, and PrismML's model card points Ollama and Docker Model Runner at it. All three hold the same ternary weights.
Does the 2B have vision and thinking mode?
Yes to both: it takes image input through a separate 0.36 GB Q8_0 projector file, and like the rest of the Qwen3.5 family it thinks by default. PrismML recommends allowing at least 8,192 output tokens with thinking on, and switching thinking off for short answers.