309B total / 15B active (MoE); 308,778,780,864 main-model parameters + 987,820,224 across three MTP blocks
256K (262,144 tokens)
MIT (model weights; GitHub repository code LICENSE is Apache-2.0)
Original text-only Base and hybrid-thinking assistant checkpoints, released in mixed FP8 precision with three lightweight dense MTP blocks. The 48-layer, width-4096 Transformer uses 39 sliding-window GQA layers (128-token window; 64 query/8 KV heads) and nine global GQA layers (64 query/4 KV heads), rather than MLA or DeepSeek Sparse Attention; the first layer has a dense FFN, while the other 47 use 256 routed experts with eight selected per token and no shared experts. Learnable attention sinks stabilize SWA; the advertised 5:1 pattern has a first-layer global-attention exception. Randomly initialized pretraining uses AdamW and 27T tokens: 22T general pretraining, 4T code-upsampled mid-training with about 5% synthetic reasoning data, and 1T context extension from 32,768 to 262,144 tokens. One MTP head is used in pretraining, then replicated and jointly trained for multi-step prediction during post-training. Post-training combines millions of thinking/non-thinking SFT examples generated by in-house specialized checkpoints, domain-specialized RL/SFT teachers, and Multi-Teacher On-Policy Distillation (MOPD), using student rollouts with token-level reverse-KL guidance and outcome rewards. Code-agent RL scales to about 120K environments; video-based web-development rewards come from an external multimodal verifier, not native visual input. Launch-reported results include AIME 2025 94.1%, GPQA-Diamond 83.7%, SWE-bench Verified 73.4%, SWE-bench Multilingual 71.7%, and LongBench V2 60.6%. Coding and search scores require external tools/agent loops; BrowseComp rises from 45.4 to 58.3 with external context management, not a larger native context. Attempt budgets and score aggregation are incompletely disclosed. The report notes weaker Base factual knowledge and a remaining gap to the strongest closed models. Three-layer MTP enables self-speculative decoding; the launch blog reports 2.0–2.6× speedups, while the joint SGLang benchmark reports about 150 tokens/s per request on H200 at batch 16 per DP rank and up to 64K input, not a hardware-independent speed. Launch API pricing of $0.10/$0.30 per million input/output tokens is separate from local inference cost. Tool templates support thinking toggles and require preserving reasoning history in multi-turn thinking/tool use.
https://github.com/XiaomiMiMo/MiMo-V2-Flash/blob/65f0e73804b030611b74235ddad9dc4ad1a6a586/paper.pdf