8.31B each (8,310,501,376; dense, tied embeddings)
32K (32,768 tokens)
Apache 2.0
Essential AI's first open-weight model family, comprising text-only Base and instruction-tuned checkpoints trained from scratch for coding, mathematics and scientific reasoning. Uses a Gemma-3-like dense decoder architecture rather than Gemma-derived weights: 32 layers, 4,096 hidden width, 16,384 MLP width, grouped-query attention with 32 query heads and 8 KV heads, tied embeddings and global attention throughout. Pretraining used 8.4T tokens at 8K context with Muon and a warmup-stable-decay learning-rate schedule; a further 380B-token mid-training stage extended context to 32K with YaRN, followed by 150B tokens of supervised fine-tuning for Instruct. Training included fill-in-the-middle code, program execution and code evolution; no released-model reinforcement-learning stage is documented. Supports multilingual code generation, code infilling and tool calls in <tool_call> tags, with demonstrated agentic coding and profiler-assisted optimization. Publisher reports 20.8% SWE-bench Verified with mini-swe-agent's bash-only scaffold, exceeding Gemini 2.0 Flash and Qwen2.5-Coder 32B Instruct under that framework, and strong BFCL, math and science results among comparable-size models. Reported HE-FIM-Python averages are 82.49% for Base and 86.21% for Instruct. Separate test-time-scaling results use pass@1/2/4/8 from eight generations at temperature 0.2, not single-response guarantees; comparator settings include GPT-OSS-20B at low reasoning effort and Qwen3-8B with thinking disabled. Post-training was deliberately limited to leave room for community specialization. Publisher notes weak factual recall, identity and knowledge-cutoff hallucinations, and a tendency to emit code on non-code prompts. December updates recommend a system prompt and temperature 0–0.2 to mitigate premature truncation; optional 128K YaRN extrapolation requires configuration changes and shows some regressions, so it is not the launch checkpoint's native context.