78.1B total / 3.46B active (MoE)
1,048,576 tokens
Apache 2.0
Open-weight English-German reasoning and tool-use model designed by Aleph Alpha for sovereign deployment, regulated industries, public administration, industrial applications, and aerospace. Kolibri-1 is a 50-layer decoder-only causal Transformer with 78,103,074,560 total parameters and 3,457,573,120 active parameters per token, activating approximately 4.4% of its parameters. Its hybrid-attention architecture contains 40 sliding-window-attention blocks with a 512-token window and ten full-attention blocks arranged at a 4:1 ratio. The model has a width of 2,560, 48 query heads, four KV heads, and 128-dimensional attention heads. Every layer contains a sparse MoE comprising 384 routed experts, of which six are selected per token, plus one shared expert. The architecture uses sandwich RMS normalization, grouped-query attention, Muon optimization, and Exact Quantile Balancing for expert routing. Kolibri was trained on approximately 24 trillion tokens across three stages: 20T tokens of pre-training at 16K context, 3.44T tokens of mid-training at 64K, and 201B tokens of long-context extension at 256K. The pre-training corpus was approximately 62.5% English, 23.9% German, and 13.6% code, including more than 2T German tokens curated from the web or generated synthetically. Its 128,000-token UniBPE vocabulary was developed to improve German morphological alignment and compression while retaining competitive English efficiency. Post-training combines bilingual supervised fine-tuning with reinforcement learning across more than 1.2 million internally curated tasks spanning reasoning, instruction following, code, retrieval, and tool use. The model supports low, medium, and high reasoning-effort levels, as well as a non-thinking mode, and exposes Hermes-style structured tool calls through Aleph Alpha's open-source vLLM plugin. Although its final long-context training used sequences up to 262,144 tokens, Aleph Alpha validated quality and serving efficiency at 1,048,576 tokens and recommends contexts at or below 262,144 for complex or throughput-sensitive workloads. Aleph Alpha reports scores of 96.9 on AIME 2025, 96.0 on AIME 2026, 84.3 on GPQA Diamond, 85.9 on LiveCodeBench v6, 92.7 on HumanEval+, 61.4 on BFCL v4, 64.5 on LongBench Pro, 69.9 on tau2-bench Retail, and 38.1 on tau3-bench Banking. Pre-training used 768 NVIDIA B200 GPUs for 21 days, followed by five days of mid-training and thirteen hours of long-context training. The released FP8 checkpoint uses float8_e4m3fn weights in 128-by-128 blocks with dynamically quantized activations while retaining embeddings, the language-model head, normalization layers, and MoE routers in BF16; its 32 weight shards total approximately 78.84 GB. A separate 32-shard BF16 checkpoint totals approximately 156.21 GB. The official model card lists a roughly 78 GB FP8 memory footprint and a minimum deployment configuration of two 80 GB A100s, two H100 SXM5 GPUs, one H200, one B200, or one B300. Both checkpoints are served through the Apache-2.0-licensed aleph-alpha-inference vLLM plugin, which provides Kolibri-specific reasoning and tool-call parsers.