← Back to models

1-bit Bonsai 8B (GGUF, MLX)

PrismML March 31, 2026

Parameters

8.19B (Qwen3-8B dense base; end-to-end 1-bit deployment format)

Context Window

65,536 tokens

License

Apache-2.0

Key Features

Open-weight 1-bit Bonsai 8B release positioned as the first commercially viable end-to-end 1-bit LLM for real on-device and edge deployment. Official PrismML materials claim true 1-bit coverage across embeddings, attention projections, MLP projections, and LM head with no higher-precision escape path, reducing deployed parameter memory to about 1.15 GB versus 16.38 GB FP16 (about 14x smaller) while remaining competitive with leading full-precision 8B-class instruct models on its benchmark suite. The release highlights major throughput and efficiency claims across runtimes and hardware, including approximately 6.2x faster token generation on RTX 4090 versus FP16 baseline and around 4x-5x lower energy per generated token on reported measurements. Model-card documentation describes cross-platform deployment paths (llama.cpp GGUF on CUDA/Metal/CPU/mobile plus MLX 1-bit for Apple Silicon), a 65,536-token context window, and an intelligence-density framing that presents strong quality-per-GB positioning for latency- and memory-constrained assistants, edge robotics, and private inference scenarios.

Paper / Source

https://github.com/PrismML-Eng/Bonsai-demo/blob/main/1-bit-bonsai-8b-whitepaper.pdf

Official Links