What is Ternary Bonsai 2 9B?
Ternary Bonsai 2 9B is a compressed vision-language model from PrismML, a start-up founded by Caltech researchers, announced on 15 October 2026 alongside 2B and 4B siblings. It is derived from Alibaba's Qwen3.5-9B with the architecture unchanged, but every language-model weight, including the embeddings and output head, is ternary: each is -1, 0 or +1 times a shared FP16 scale. The language model is about 8.95 billion parameters, plus a 0.46-billion-parameter vision tower.
The default GGUF file (PQ2_0) is 2.42 GB, against about 17.9 GB at FP16; a denser packing is 2.00 GB, and the optional Q8_0 vision projector adds 0.62 GB. It keeps the base model's 262,144-token context, thinking mode and tool calling, and is released under Apache 2.0. The GGUF repository has about 317,000 downloads in the past month.
The two smallest packings need PrismML's llama.cpp fork; stock llama.cpp runs only the 2.56 GB Q2_0 file. An MLX build covers Apple devices.
Is Ternary Bonsai 2 9B worth it?
Bonsai 2 9B makes a strong case for what fits in about 2.5 GB, provided your tasks are coding, maths or following instructions. PrismML's own comparison with the full-precision Qwen3.5-9B shows near-parity there: 89.02 against 87.80 on HumanEval+, 90.02 against 89.65 on IFEval and 98.60 against 99.00 on MATH500. All of these are self-reported, with thinking on.
The losses fall where compression usually bites. GPQA Diamond drops from 82.53 to 69.9, Tau2Bench agent tasks from 85.25 to 73.02, and the document-parsing score on OmniDocBench from 88.73 to 73.8. That last figure matters because a vision model on a phone is often asked to read receipts and forms, and here it keeps only 83% of the base model's ability.
One independent test on a 6 GB RTX 3060 laptop GPU backs up the footprint: 2,518 MiB of video memory at a 4K context and 83 tokens a second sustained, with up to a 32K context fitting in 6 GB. The same tester found the weakness to watch in agents: Bonsai scored 79.6 on BFCL's irrelevance test, the lowest of the models tried, meaning it is the most likely to call a tool when it should not.
The toolchain is the practical cost. The 2.00 GB and 2.42 GB files load only in PrismML's llama.cpp fork, the BF16 vision projector can crash llama-server under concurrent image requests, and thinking mode can spend 3,000 tokens on a simple prompt, so on a phone at PrismML's quoted 34 tokens a second a reasoning answer can take well over a minute.
Where it earns its place
An iOS developer adding an offline assistant to a note-taking app
Answer questions about notes and photos without a server
Ship the MLX 2-bit build; PrismML quotes up to 34 tokens a second on an iPhone 18 Pro Max. Turn thinking off for chat replies, since thinking can use 3,000 or more tokens before answering.
A platform engineer with a fleet of laptops that have 6 GB GPUs
Run a local coding and shell helper with no data leaving the machine
Serve the PQ2_0 file with llama-server from the PrismML fork. One independent test measured 2,518 MiB of video memory at 4K context and 83 tokens a second on an RTX 3060 laptop GPU, with contexts up to 32K fitting in 6 GB.
A product manager at a field-service company
Decide whether technicians can read equipment labels and manuals on-device
Feasible for labels and short text, with --image-min-tokens 1024 for small print, but PrismML reports 73.8 on OmniDocBench against 88.73 for the base model, so dense manuals and forms are better sent to a dedicated OCR model when a connection exists.
Before you commit
- Setup
- Download a prebuilt release of the PrismML-Eng/llama.cpp fork, then run llama-cli or llama-server with the PQ2_0 GGUF and the Q8_0 mmproj for images; PrismML-Eng/Bonsai-demo holds the tested setups.
- Integrations
- llama.cpp (CUDA, Metal, CPU) via PrismML's fork, an OpenAI-compatible llama-server, MLX through mlx-vlm 0.7.8 or newer, an optional MTP draft head for speculative decoding, and a hosted endpoint on the Darkbloom console.
- Data and privacy
- Apache 2.0, following the Qwen3.5-9B base model, with no revenue cap or use restrictions beyond the licence. Running locally, prompts and images stay on the device.
- Cost at scale
- Free to run on hardware you own: the 2.00 GB PTQ1_0 file is the smallest and the 2.68 GB MTP file adds speculative decoding. PrismML quotes up to 179 tokens a second on an M5 Max and up to 5 times the throughput of the full-precision model on an RTX 4090 and M5 Pro. Darkbloom's prices are not published.
- Limits
- 262,144-token context, though a 6 GB GPU fits about 32K. Allow at least 8,192 output tokens with thinking on, avoid greedy decoding with thinking, and use the Q8_0 vision projector rather than BF16 under concurrent image requests.
What's good
- 2.42 GB file for a 9B-class model with image input
- Matches Qwen3.5-9B on HumanEval+, IFEval and MATH500 (self-reported)
- 262,144-token context and tool calling kept from the base model
- Apache 2.0
What's not
- GPQA Diamond 69.9 against 82.53 for the base model
- Document parsing keeps only 83% of Qwen3.5-9B (self-reported)
- Smallest packings need PrismML's llama.cpp fork
- Calls tools unprompted more often than rivals in one independent test
Best for
Developers shipping a local assistant or coding helper on phones, laptops or 6 GB GPUs where memory matters more than the last points of reasoning.
Ternary Bonsai 2 9B alternatives
Ternary Bonsai 2 9B: common questions
- Is Ternary Bonsai 2 9B worth it?
- Bonsai 2 9B puts a capable nine-billion-parameter vision model in the memory of a phone, and gives back most of what it costs on reasoning-heavy work. We rate it 4.0 out of 5.
- How much does Ternary Bonsai 2 9B cost?
- Free under Apache 2.0. Hosted on the Darkbloom console; prices not published.
- What are the downsides of Ternary Bonsai 2 9B?
- GPQA Diamond 69.9 against 82.53 for the base model; document parsing keeps only 83% of Qwen3.5-9B (self-reported); smallest packings need PrismML's llama.cpp fork; calls tools unprompted more often than rivals in one independent test.
- Who is Ternary Bonsai 2 9B best for?
- Developers shipping a local assistant or coding helper on phones, laptops or 6 GB GPUs where memory matters more than the last points of reasoning.
- How much does Ternary Bonsai 2 9B cost for a team or in production?
- Free to run on hardware you own: the 2.00 GB PTQ1_0 file is the smallest and the 2.68 GB MTP file adds speculative decoding. PrismML quotes up to 179 tokens a second on an M5 Max and up to 5 times the throughput of the full-precision model on an RTX 4090 and M5 Pro. Darkbloom's prices are not published.
- Is Ternary Bonsai 2 9B safe for company data?
- Apache 2.0, following the Qwen3.5-9B base model, with no revenue cap or use restrictions beyond the licence. Running locally, prompts and images stay on the device.
- What are the best Ternary Bonsai 2 9B alternatives?
- The closest alternatives we have reviewed are Gemma 4, Ministral 3 (3B, 8B and 14B), MiniCPM5-2B and gpt-oss-20b.
Answers checked Oct 28, 2026.
Make Ternary Bonsai 2 9B? Show this rating on your site
