CanItRun Logocanitrun.

Bonsai 2 9B

Bonsai 2 9B doesn't use the standard quantization ladder; it ships as fixed builds needing 2.5 GB at Ternary (PTQ1_0) or 3.0 GB at Ternary (PQ2_0) or 3.2 GB at Ternary (Q2_0). 117 GPUs we track can run at least one build fully in VRAM at 8k context.

117 GPUs run this natively · 1 with CPU offload

PrismML9B params256k contextApache 2.0Commercial use ok
Ternary (PQ2_0) total
3.0 GB
at 8k context
Smallest GPU
4 GB
Intel Arc A310 4GB, at Ternary (PQ2_0)
KV cache, full context
8.6 GB
8 of 32 layers cache
Inputs
Text · Image
Apache 2.0, released October 2026
Knowledge retention
93.0%
PrismML's own suite vs Qwen3.5-9B
Coding retention
100.1%
HumanEval+ 89.02 vs 87.80

Bonsai 2 9B vs Qwen 3.5 9B: size & quality

Same weights as Qwen 3.5 9B, requantized to a fraction of the size.

Weights size

Qwen 3.5 9B FP16
18.0 GB
Qwen 3.5 9B Q4_K_M
5.5 GB
Bonsai Ternary (PTQ1_0)
2.0 GB
Bonsai Ternary (PQ2_0)
2.4 GB
Bonsai Ternary (Q2_0)
2.6 GB

Bonsai 2 9B is a 9B parameter dense model developed by PrismML. Released October 15, 2026 by PrismML as the largest of three small models (2B, 4B, 9B) that extend the Bonsai 2 line down from the 27B: a ternary requantization of Alibaba's Qwen3.5-9B, same architecture and layer layout, not a new pretrain or finetune. Apache 2.0, text plus image input, and the 262,144-token context window inherited from the base model. Ships as three packings of the same weights: PTQ1_0 (2.00 GB, dense trits), PQ2_0 (2.42 GB, PrismML's default) and Q2_0 (2.56 GB, the one upstream llama.cpp b11555 or newer can read); an MTP variant of the PQ2_0 file adds a draft head for speculative decoding.

To run Bonsai 2 9B locally: At 2.42 GB of weights (3.0 GB in total at 8k context) it is small enough for most GPUs this site tracks, a base-tier MacBook, or a recent phone; PrismML reports up to 34 tok/s on an iPhone 18 Pro Max and up to 124 tok/s on an RTX 4090 or M5 Pro, its own numbers. The catch is context: the full 262,144-token window needs 8.6 GB of KV cache, several times the weights, so long-context use is bounded by the cache, not the download. Stock llama.cpp only reads the Q2_0 file; PTQ1_0, PQ2_0 and the MTP variant need PrismML's llama.cpp fork.

PrismML's own 19-benchmark table, run on this build and on full-precision Qwen3.5-9B in the same harness, shows it matching or edging past its base on coding (HumanEval+ 89.02 vs 87.80) and instruction following (IFEval 90.02 vs 89.65), and landing within a point on math (MATH500 98.6 vs 99.0). Knowledge and long agentic work take the real hit: GPQA Diamond 69.9 vs 82.53 (84.7% retained) and Tau2Bench 73.02 vs 85.25 (85.7%). All of these are PrismML's own figures, not yet reproduced by an independent leaderboard.

Why the context window, not the 2 GB download, sets the ceiling

Ternary quantization shrinks the weights and nothing else: the KV cache stays 16-bit. That is easy to miss at a short context, where the cache is a rounding error, and impossible to miss at the model's full window.

01020304032k128k192k256k8.6 GBBonsai 2 9B34.4 GBall-attention 9B
Bonsai 2 9B: 8 of 32 layers cacheThe same model if all 32 layers used full attention

KV cache only, at FP16. Weights and activation overhead sit on top of these figures.

Only 8 of the 32 layers keep a KV cache (the other 24 are Gated DeltaNet layers with a fixed-size state), so the full 262,144-token window costs 8.6 GB of cache, against 34.4 GB if every layer cached. That is still more than three and a half times the 2.42 GB of PQ2_0 weights: a full-window session needs 12.3 GB in total, versus 3.0 GB at 8k context. If a long context is the point, size the GPU for the cache first and treat the small download as a bonus.

Available builds

Unlike models with a full FP32–Q2_K ladder, Bonsai 2 9B ships in 3 pre-baked low-bit builds only. The table below uses 8k of context as its baseline.

BuildBits/weightWeightsKV cacheTotalQuality vs FP16
Ternary (PTQ1_0)1.782.0 GB0.27 GB2.5 GBN/A
Ternary (PQ2_0)2.162.4 GB0.27 GB3.0 GBN/A
Ternary (Q2_0)2.282.6 GB0.27 GB3.2 GBN/A

*Self-reported by the vendor, not yet cross-checked against an independent leaderboard. KV cache numbers assume 8k context and FP16 precision.

Benchmarks

GPQA Diamond
69.9
IFEval
90.0

Estimated, not measured: derived from Qwen 3.5 9B's real score times the best build's vendor-reported quality retention (see the chart above), not independently verified, and not a leaderboard result for Bonsai 2 9B itself.

PrismML's own benchmark table: what ternary weights cost a 9B

PrismML's launch table runs the ternary 9B and full-precision Qwen3.5-9B through the same harness, so each row is a like-for-like comparison. A sample of eight of its 19 benchmarks, spanning every category it reports:

BFCLv3 (tool calling)
Bonsai 2 9B
71.8
Qwen 3.5 9B base
73.8
Tau2Bench (agentic)
Bonsai 2 9B
73.0
Qwen 3.5 9B base
85.3
MATH500
Bonsai 2 9B
98.6
Qwen 3.5 9B base
99.0
IFEval
Bonsai 2 9B
90.0
Qwen 3.5 9B base
89.7
HumanEval+
Bonsai 2 9B
89.0
Qwen 3.5 9B base
87.8
MMLU-Redux
Bonsai 2 9B
87.0
Qwen 3.5 9B base
91.1
GPQA Diamond
Bonsai 2 9B
69.9
Qwen 3.5 9B base
82.5
Omni_doc (vision)
Bonsai 2 9B
73.8
Qwen 3.5 9B base
88.7

PrismML, Bonsai 2 family announcement (prismml.com/news/bonsai-2-family), 15 October 2026. PrismML's own harness, run on the ternary build and on full-precision Qwen3.5-9B side by side; not yet listed on any independent leaderboard or independently reproduced.

Coding, instruction following and math hold at 99 to 101% of the base model on the rows shown, which is the part of the ternary pitch that checks out. The losses sit where a model has to know things or stay on track over many steps: GPQA Diamond keeps 84.7% (69.9 vs 82.53), Tau2Bench 85.7% and document-style vision (Omni_doc) 83.2%. PrismML reports 93.0% retention across its knowledge and reasoning category overall. These are PrismML's own numbers; the same harness gives Qwen3.5-9B a GPQA Diamond of 82.53, a little above the 81.7 Alibaba itself reports, so do not compare these figures to other models' published scores directly.

GPUs that run Bonsai 2 9B natively (117)

Show 112 more
Plus 1 GPU that runs it with CPU offload (slower)

No GPU? Rent an RTX 3080 10GB from $0.17/hr ↗

Prices as of Oct 13, 2026. Rates change; check RunPod for current pricing. Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Notes

Third-party ternary requantization of Qwen3.5-9B by PrismML, same architecture, not a retrain, and part of the Bonsai 2 small-model release of 15 October 2026 (2B, 4B, 9B). Doesn't use the standard FP32-Q2_K ladder shown for other models; it ships as three fixed packings of the same ternary weights (see the quantization table on this page): PTQ1_0 and PQ2_0 need PrismML's own llama.cpp fork, while Q2_0 runs on upstream llama.cpp b11555 or newer. File sizes are the language model only; the Q8_0 vision projector adds 0.62 GB. GPQA Diamond and IFEval are PrismML's own measurements of this build in its in-house harness, not independently verified.

Hugging Face ↗Released 2026-10-15

Compare Bonsai 2 9B with other models

How to run Bonsai 2 9B locally

PrismML's llama.cpp fork (PQ2_0 packing, PrismML's own sampling)

./llama-server -m Ternary-Bonsai-2-9B-PQ2_0.gguf \
  --mmproj Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf \
  -ngl 99 -fa on -c 32768 --jinja --port 8080 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 \
  --presence-penalty 1.0 --repeat-penalty 1.0 --repeat-last-n 64

llama.cpp

# Build PrismML's llama.cpp fork (needed for PTQ1_0 and PQ2_0; Q2_0 also runs on upstream b11555+)
git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON   # drop the flag on macOS (Metal is default)
cmake --build build -j

# Download the weights and the Q8_0 vision projector (not the BF16 one, it can crash llama-server under concurrent image requests)
hf download prism-ml/Ternary-Bonsai-2-9B-gguf Ternary-Bonsai-2-9B-PQ2_0.gguf Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf --local-dir .

# Optional: MTP speculative decoding (load this file INSTEAD of the plain PQ2_0, 2.68 GB)
./llama-server -m Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf -ngl 99 -fa on \
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0

LM Studio: Stock LM Studio bundles upstream llama.cpp, so only the Q2_0 file (2.56 GB) is a candidate and it needs a runtime based on llama.cpp b11555 or newer; PTQ1_0 and PQ2_0 will be rejected. On Apple Silicon the MLX build (prism-ml/Ternary-Bonsai-2-9B-mlx-2bit, 3.46 GB on disk, mlx-vlm 0.7.8 or newer) is the other supported route.

Why this quantization? PQ2_0 is the packing PrismML's own run commands use: 2.42 GB of weights and 3.0 GB in total at 8k context, a small step up from the 2.00 GB PTQ1_0 file (2.5 GB total) that stores each weight as a dense trit but needs the same fork. Q2_0 (2.56 GB, 3.2 GB total) is the one to pick only if you need stock upstream llama.cpp. All three hold the same ternary weights, so quality does not differ between them; the choice is file size against runtime support. Budget for context before weights: the full 262,144-token window adds 8.6 GB of KV cache, so set -c explicitly instead of -c 0.

Who is Bonsai 2 9B for?

Anyone who needs a vision-capable, tool-calling 9B-class model on hardware that cannot hold the standard 5.5 GB Q4_K_M of Qwen 3.5 9B comfortably: small GPUs, base MacBooks, phones, or an always-on box where memory is shared with other services.

Best for

  • On-device assistants and agents on phones and thin laptops (PrismML reports up to 34 tok/s on an iPhone 18 Pro Max)
  • Coding help and instruction-following tasks, where PrismML's own table shows it level with Qwen3.5-9B
  • Image understanding and OCR-style tasks through the Q8_0 vision projector
  • A small, fast tool-calling model behind --jinja's OpenAI-style tool_calls

Not ideal for

  • Knowledge-heavy or long agentic work, where PrismML's own GPQA Diamond (84.7%) and Tau2Bench (85.7%) retention shows the biggest drops
  • Stock Ollama or LM Studio users who do not want to build PrismML's llama.cpp fork (only the Q2_0 file runs upstream)
  • Anyone who needs benchmark claims independently verified before adopting a model; released the same day this page was written
  • Very long contexts on small cards: the 262,144-token KV cache alone is 8.6 GB

So should you run it over a standard Qwen 3.5 9B?

If memory is the hard constraint, yes. The PQ2_0 build needs 2.42 GB of weights and 3.0 GB in total at 8k context, against 5.5 GB of weights and 6.4 GB in total for Qwen 3.5 9B at Q4_K_M, and PrismML's own numbers show coding and instruction following holding at the base model's level. If you have an 8GB card and your work leans on knowledge questions or long agent loops, the standard Q4_K_M is the safer pick: Bonsai's GPQA Diamond and Tau2Bench retention are 84.7% and 85.7% in PrismML's own table, and the Q4_K_M file runs on stock llama.cpp, Ollama and LM Studio without a fork. None of PrismML's figures have independent verification yet.

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Bonsai 2 9B?
Bonsai 2 9B doesn't use the standard quantization ladder; it ships as 3 fixed builds: Ternary (PTQ1_0) (2.5 GB), Ternary (PQ2_0) (3.0 GB), and Ternary (Q2_0) (3.2 GB). These figures assume 8k of context; VRAM scales linearly with context length due to the KV cache.
How many parameters does Bonsai 2 9B have?
Bonsai 2 9B has 9 billion parameters, the same count as the base model it's derived from, since this release is a post-training requantization rather than a retrain.
What quantization levels does Bonsai 2 9B come in?
Bonsai 2 9B skips the usual FP16-to-Q2_K ladder entirely. It's only available as: Ternary (PTQ1_0) at 1.78 bits/weight (2.5 GB total); Ternary (PQ2_0) at 2.16 bits/weight (3.0 GB total); Ternary (Q2_0) at 2.28 bits/weight (3.2 GB total).
Can Bonsai 2 9B run on a small GPU?
Yes. The Ternary (PTQ1_0) build needs only 2.5 GB, which fits on most 8 GB+ GPUs, entry-level Apple Silicon, and even high-end phones per the vendor's own claims.
What GPU do I need to run Bonsai 2 9B locally?
Every build is small enough for almost any GPU released in the last several years. The Ternary (Q2_0) build (3.2 GB) needs a bit more headroom for better quality, but all of them fit comfortably on an 8 GB card or larger.
Can I run Bonsai 2 9B in the cloud?
Yes. The cheapest RunPod setup we found that holds Bonsai 2 9B entirely in GPU memory at 8k context is 1× RTX 3080 10GB at Ternary (Q2_0), at roughly 162.6 tokens per second and from $0.17 per hour for the whole pod. Prices as of Oct 13, 2026; RunPod rates change, so check their pricing page for the current figure.
Which Bonsai 2 9B file should I download?
For PrismML's own build of llama.cpp, the PQ2_0 file (2.42 GB) is the default its docs and model card use; PTQ1_0 (2.00 GB) is the smallest and needs the same fork. Take Q2_0 (2.56 GB) only if you need to run on stock upstream llama.cpp b11555 or newer. All three hold the same ternary weights. Add the 0.62 GB Q8_0 vision projector if you want image input.
Can I run Bonsai 2 9B at its full 262,144-token context?
On a large enough card, yes, but the KV cache alone is 8.6 GB at that length (12.3 GB in total with the PQ2_0 weights), more than three times the weights themselves, so an 8GB card will not hold the full window even though the download is only 2.4 GB. Set -c explicitly in llama.cpp rather than -c 0, which loads the full window.