Bonsai 27B: On-Device AI Reasoning

Bonsai 27B, released by PrismML in July 2026, is a multimodal ultra-low-bit model derived from Qwen3.6-27B. It is designed to enable powerful language, vision, and agentic reasoning workflows on devices where full-precision 27B models cannot practically run. Offering two compression variants, it provides a trade-off between footprint and fidelity without relying on cloud infrastructure for most use cases.

Key Features

• Low-bit compression variants:
  • Ternary Bonsai 27B: Uses ternary weights ({−1, 0, +1}) with FP16 group-wise scaling, yielding about 1.71 bits per weight and a deployed footprint of roughly 5.9 GB. Aims for higher quality on laptops or GPUs.
  • 1-bit Bonsai 27B: Uses binary weights ({−1, +1}), averaging 1.125 bits per weight and occupies about 3.9 GB — compact enough to run on high-end phones. Both include FP16/4-bit vision towers to support multimodal input like screenshots, documents, or camera feeds.
• Multimodal context & reasoning: Bonsai 27B supports vision and language input, structured tool calls, agentic workflows, and sustained multi-step reasoning. It also ships with a full context window of 262,000 tokens, enabling long document analysis, multi-turn conversation, and other prolonged‐interaction tasks.
• Performance retention & benchmarks: Across a 15-benchmark suite covering math, coding, knowledge, vision, tool-use, and instruction following, Ternary Bonsai retains around 95% of the full-precision FP16 baseline; the 1-bit variant retains about 90%. Math and coding degrade minimally (only a few points), while agentic workflows, vision, and instruction tasks show wider drops in accuracy.
• On-device and hybrid deployment: The 1-bit model enables full 27B-class reasoning on a phone (e.g. iPhone 17 Pro Max), fitting within the model memory constraints of ~6 GB, including activations and KV cache. Ternary builds target laptop- and GPU-class hardware. Execution is supported via custom low-bit kernels on platforms like Apple MLX/Metal and CUDA, allowing for offline, private, or latency-sensitive settings.

Who is it for?

• Business owners, product teams, and professionals who need advanced AI capabilities (reasoning, tool orchestration, multimodal understanding) under strict latency or privacy constraints, such as on-device assistants, offline workflows, or embedded systems.
• Developers and researchers who want to experiment with large-model performance locally without cloud-only dependency, and need to benchmark trade-offs between model size, power, and capability.
• Users with hardware constraints, such as smartphone-class devices or laptops with limited GPU/CPU resources, who want to deploy or test 27B-class models in constrained environments.

Pricing

Bonsai 27B (both 1-bit and ternary variants) is available under the open-source Apache 2.0 license for free download starting July 14, 2026. PrismML provides a time-limited developer preview API for exploring its capabilities. There is no subscription or usage fee stated in the official announcement.

Final Thoughts

Bonsai 27B marks a significant technical shift: it pushes 27B-class model performance into the realm of phones and modest laptops without wholesale reliance on the cloud. For math and coding tasks, the trade-offs are mild; for vision and complex multi-step workflows, there is noticeable degradation. Decision-makers should match the variant to their hardware and usage needs: use 1-bit where footprint is critical, or ternary for better overall fidelity. Its Apache license makes experimentation and internal deployment low-risk, though the lack of published independent benchmarks in some categories means evaluations for mission-critical work should be done in-house. For organizations balancing capability, privacy, and cost, Bonsai 27B offers a compelling local-first option.

Visit the official website for more.

Keep up to date with our stories on LinkedIn, Twitter, Facebook and Instagram.