Bonsai 27B
Bonsai 27B doesn't use the standard quantization ladder — it ships as fixed builds needing 6.1 GB at 1-bit (Q1_0) or 9.9 GB at Ternary (Q2_0). 75 GPUs we track can run at least one build fully in VRAM at 8k context.
75 GPUs run this natively · 5 with CPU offload
Bonsai 27B vs Qwen 3.6 27B: size & quality
Same weights as Qwen 3.6 27B, requantized to a fraction of the size.
Weights size
Quality retained
Quality retained is Qwen 3.6 27B's own FP16 output as the 100% reference, against PrismML's vendor-reported retention for each build — not yet verified by an independent leaderboard.
Bonsai 27B is a 27B parameter dense model developed by PrismML. Released July 2026 by PrismML as a proprietary low-bit requantization of Alibaba's Qwen3.6-27B — identical underlying weights, not a new pretrain or finetune. Two fixed builds, both Apache 2.0 (see the size chart above).
To run Bonsai 27B locally: Small enough for 8GB GPUs, base-tier Apple Silicon, or high-end phones. PrismML reports ~163 tok/s (1-bit) and ~134 tok/s (Ternary) on an RTX 5090, and ~87/58 tok/s on an M5 Max — vendor-reported, not independently verified.
Quality losses concentrate in agentic tool-calling and instruction-following rather than math (see the quality chart above). Not yet listed on any independent leaderboard.
Available builds
Bonsai 27B doesn't use the standard FP32–Q2_K quantization ladder — it's a fixed low-bit release with exactly 2 downloadable builds. Sizes below assume 8k context.
| Build | Bits/weight | Weights | KV cache | Total | Quality vs FP16 |
|---|---|---|---|---|---|
| 1-bit (Q1_0) | 1.125 | 3.8 GB | 1.61 GB | 6.1 GB | ~89.5%* |
| Ternary (Q2_0) | 1.58 | 7.2 GB | 1.61 GB | 9.9 GB | ~94.6%* |
*Vendor-reported on their own benchmark suite — not yet verified by an independent leaderboard. KV cache shown at 8k context (FP16).
Benchmarks
Independent benchmark scores for Bonsai 27B are not yet listed on the Open LLM Leaderboard v2. Third-party requantization of Qwen3.6-27B by PrismML — same weights, not a retrain. Doesn't use the standard FP32-Q2_K ladder shown for other models; it ships only as two fixed sub-2-bit builds (see the quantization table on this page). PrismML's own benchmark suite reports 89.5%–94.6% of FP16 quality retained across the two builds; not yet independently verified by a third party. See the HuggingFace model card for reported evaluations.
GPUs that run Bonsai 27B natively (75)
- NVIDIA RTX 5090Ternary (Q2_0) · 248.9 t/s
- NVIDIA RTX 5080Ternary (Q2_0) · 133.3 t/s
- NVIDIA RTX 5070 TiTernary (Q2_0) · 124.4 t/s
- NVIDIA RTX 5070Ternary (Q2_0) · 93.3 t/s
- NVIDIA RTX 5060 Ti 16GBTernary (Q2_0) · 62.2 t/s
- NVIDIA RTX 50601-bit (Q1_0) · 117.9 t/s
- NVIDIA RTX 50501-bit (Q1_0) · 84.2 t/s
- NVIDIA RTX 4090Ternary (Q2_0) · 140 t/s
- NVIDIA RTX 4080Ternary (Q2_0) · 99.6 t/s
- NVIDIA RTX 4070 TiTernary (Q2_0) · 70 t/s
- NVIDIA RTX 4070Ternary (Q2_0) · 70 t/s
- NVIDIA RTX 4060 Ti 16GBTernary (Q2_0) · 40 t/s
- NVIDIA RTX 40601-bit (Q1_0) · 71.6 t/s
- NVIDIA RTX 3090Ternary (Q2_0) · 130 t/s
- NVIDIA RTX 3090 TiTernary (Q2_0) · 140 t/s
- NVIDIA RTX 3080 10GB1-bit (Q1_0) · 200 t/s
- NVIDIA RTX 3060 12GBTernary (Q2_0) · 50 t/s
- NVIDIA H100 80GBTernary (Q2_0) · 465.3 t/s
- NVIDIA A100 80GBTernary (Q2_0) · 283.2 t/s
- NVIDIA A100 40GBTernary (Q2_0) · 216 t/s
- NVIDIA L40STernary (Q2_0) · 120 t/s
- NVIDIA RTX A6000Ternary (Q2_0) · 106.7 t/s
- NVIDIA RTX 4000 AdaTernary (Q2_0) · 44.4 t/s
- NVIDIA RTX 4500 AdaTernary (Q2_0) · 60 t/s
- NVIDIA RTX 5000 AdaTernary (Q2_0) · 80 t/s
- NVIDIA RTX 6000 AdaTernary (Q2_0) · 133.3 t/s
- NVIDIA RTX Pro 6000Ternary (Q2_0) · 186.7 t/s
- NVIDIA DGX Spark (128GB)Ternary (Q2_0) · 37.9 t/s
- AMD Radeon RX 7900 XTXTernary (Q2_0) · 133.3 t/s
- AMD Radeon RX 7900 XTTernary (Q2_0) · 111.1 t/s
- AMD Radeon RX 7900 GRETernary (Q2_0) · 80 t/s
- AMD Radeon RX 6800 XTTernary (Q2_0) · 71.1 t/s
- AMD Radeon PRO W7800Ternary (Q2_0) · 80 t/s
- AMD Radeon PRO W7900Ternary (Q2_0) · 120 t/s
- AMD Instinct MI300XTernary (Q2_0) · 736.1 t/s
- AMD Radeon AI Pro 9700 32GBTernary (Q2_0) · 88.9 t/s
- AMD Strix Halo (128GB)Ternary (Q2_0) · 35.6 t/s
- AMD Strix Halo (96GB)Ternary (Q2_0) · 35.6 t/s
- AMD Strix Halo (64GB)Ternary (Q2_0) · 35.6 t/s
- Apple M5 Max (128GB)Ternary (Q2_0) · 85.3 t/s
- Apple M5 Max (64GB)Ternary (Q2_0) · 85.3 t/s
- Apple M5 Max (48GB)Ternary (Q2_0) · 85.3 t/s
- Apple M5 Pro (48GB)Ternary (Q2_0) · 42.6 t/s
- Apple M4 Ultra (384GB)Ternary (Q2_0) · 151.7 t/s
- Apple M4 Ultra (192GB)Ternary (Q2_0) · 151.7 t/s
- Apple M4 Max (128GB)Ternary (Q2_0) · 75.8 t/s
- Apple M4 Max (96GB)Ternary (Q2_0) · 75.8 t/s
- Apple M4 Max (64GB)Ternary (Q2_0) · 75.8 t/s
- Apple M4 Max (48GB)Ternary (Q2_0) · 75.8 t/s
- Apple M4 Pro (48GB)Ternary (Q2_0) · 37.9 t/s
- Apple M3 Ultra (512GB)Ternary (Q2_0) · 113.8 t/s
- Apple M3 Ultra (256GB)Ternary (Q2_0) · 113.8 t/s
- Apple M3 Ultra (96GB)Ternary (Q2_0) · 113.8 t/s
- Apple M3 Max (128GB)Ternary (Q2_0) · 55.6 t/s
- Apple M3 Max (96GB)Ternary (Q2_0) · 55.6 t/s
- Apple M3 Max (64GB)Ternary (Q2_0) · 55.6 t/s
- Apple M3 Max (48GB)Ternary (Q2_0) · 55.6 t/s
- Apple M2 Ultra (384GB)Ternary (Q2_0) · 111.1 t/s
- Apple M2 Ultra (192GB)Ternary (Q2_0) · 111.1 t/s
- Apple M2 Max (96GB)Ternary (Q2_0) · 55.6 t/s
- Apple M2 Max (64GB)Ternary (Q2_0) · 55.6 t/s
- Apple M1 Ultra (128GB)Ternary (Q2_0) · 111.1 t/s
- Apple M1 Ultra (64GB)Ternary (Q2_0) · 111.1 t/s
- Apple M1 Max (64GB)Ternary (Q2_0) · 55.6 t/s
- Intel Arc B580 12GBTernary (Q2_0) · 63.3 t/s
- Intel Arc B570 10GB1-bit (Q1_0) · 100 t/s
- Intel Arc Pro B70 24GBTernary (Q2_0) · 63.3 t/s
- Intel Arc Pro B60 24GBTernary (Q2_0) · 52.8 t/s
- Intel Arc A770 16GBTernary (Q2_0) · 77.8 t/s
- Intel Arc A770 8GB1-bit (Q1_0) · 134.7 t/s
- Intel Arc A750 8GB1-bit (Q1_0) · 134.7 t/s
- Intel Arc A580 8GB1-bit (Q1_0) · 134.7 t/s
- Intel Arc Pro A60 12GBTernary (Q2_0) · 53.3 t/s
- Intel Data Center GPU Max 1550Ternary (Q2_0) · 455 t/s
- Intel Data Center GPU Max 1100Ternary (Q2_0) · 170.7 t/s
Plus 5 GPUs that run it with CPU offload (slower)
- Intel Arc A380 6GBTernary (Q2_0) · 6.5 t/s
- Intel Arc A310 4GBTernary (Q2_0) · 4.3 t/s
- Intel Arc Pro A50 6GBTernary (Q2_0) · 6.7 t/s
- Intel Arc Pro A40 6GBTernary (Q2_0) · 6.7 t/s
- CPU only (system RAM)Ternary (Q2_0) · 1.5 t/s
Notes
Third-party requantization of Qwen3.6-27B by PrismML — same weights, not a retrain. Doesn't use the standard FP32-Q2_K ladder shown for other models; it ships only as two fixed sub-2-bit builds (see the quantization table on this page). PrismML's own benchmark suite reports 89.5%–94.6% of FP16 quality retained across the two builds; not yet independently verified by a third party.
GPUs mentioned
Frequently asked questions
- What are the VRAM requirements for Bonsai 27B?
- Bonsai 27B doesn't use the standard quantization ladder — it ships as 2 fixed builds: 1-bit (Q1_0) (6.1 GB) and Ternary (Q2_0) (9.9 GB). These figures assume a 8k context window; VRAM scales linearly with context length due to the KV cache.
- How many parameters does Bonsai 27B have?
- Bonsai 27B has 27 billion parameters — the same count as the base model it's derived from, since this release is a post-training requantization rather than a retrain.
- What quantization levels does Bonsai 27B come in?
- Bonsai 27B skips the usual FP16-to-Q2_K ladder entirely. It's only available as: 1-bit (Q1_0) at 1.125 bits/weight (6.1 GB total, ~89.5% of FP16 quality per the vendor); Ternary (Q2_0) at 1.58 bits/weight (9.9 GB total, ~94.6% of FP16 quality per the vendor).
- Can Bonsai 27B run on a small GPU?
- Yes. The 1-bit (Q1_0) build needs only 6.1 GB, which fits on most 8 GB+ GPUs, entry-level Apple Silicon, and even high-end phones per the vendor's own claims.
- What GPU do I need to run Bonsai 27B locally?
- Either build is small enough for almost any GPU released in the last several years. The Ternary (Q2_0) build (9.9 GB) needs a bit more headroom for better quality, but both fit comfortably on an 8 GB card or larger.