12B total / 2.5B active (MoE)
128K tokens
Apache 2.0
Official open-weight Mellum 2 family release covering base, instruct, and thinking checkpoints for software-engineering-focused text generation. JetBrains describes Mellum 2 as a 12B-parameter Mixture-of-Experts model with 2.5B active parameters per token, positioned as a successor to the earlier 4B dense Mellum and specialized for code generation and editing, debugging, multi-step reasoning, tool use and function calling, agentic coding, and conversational programming assistance. The Mellum2 Technical Report says the architecture uses 64 experts with 8 active per token, Grouped-Query Attention with 4 KV heads, Sliding Window Attention on three of every four layers, and a single Multi-Token Prediction head that also serves as a built-in draft model for speculative decoding. Training materials describe roughly 10.6T pretraining tokens, a three-phase curriculum that increasingly emphasizes curated code and math data, extension of the base model to a 128K context window via layer-selective YaRN, and a two-stage post-training pipeline of supervised fine-tuning followed by RLVR. The released variants separate direct-answer and explicit-reasoning behavior into Instruct and Thinking checkpoints respectively, and JetBrains also published same-day SFT intermediate checkpoints on Hugging Face alongside the main family release.