309B total / 15.5B active (MoE)
1,048,576 tokens
MIT
Open-weight coding and AI-R&D model produced by continuing from Xiaomi's MiMo-V2.5 base and replacing its global-attention layers with DeepSeek Sparse Attention. The 48-layer, 4,096-dimensional MoE contains 256 routed experts with eight selected per token and uses a hybrid stack of 39 Sliding-Window Attention layers and nine DeepSeek Sparse Attention layers, organized primarily as eight six-layer modules with five SWA layers followed by one DSA layer and an additional DSA layer at the start of the network. No layer uses full attention. SWA operates over a 128-token window, while DSA uses a 16-head lightweight indexer to select the top 2,048 historical tokens for backbone attention. DSA replaces the original MLA formulation with grouped-query attention using four KV groups; both attention mechanisms incorporate sink bias. NaiveAI reports that the redesigned indexer reduces selection latency by 44% relative to the original DSA implementation while retaining agent-task performance at the native 1,048,576-token context length. Continued training totaled 3.25 trillion tokens: 50B tokens of indexer warmup, 3T tokens of sparse-attention training, and 200B tokens of learning-rate decay. Official evaluations used Claude Code 2.1.207 with basic file and Bash tools, the full 1M-token window, temperature 1.0, and top-p 0.95. NaiveAI reports scores of 67.8 on DeepSWE v1.1, 86.7 on Terminal-Bench 2.1, 73.6 on SWE-bench Pro, 71.9 on NL2Repo-Bench, 78.2 on FrontierSWE v1, 32.4 on Agents' Last Exam CLI, 37.5 on PostTrainBench, 73.7% on MLE-bench-30, and 63.2 on PaperBench. NaiveRT, the associated deterministic inference runtime, combines mega-kernel fusion, Programmatic Dependent Launch, GPU-driven scheduling, communication overlap, and speculative decoding with a five-layer DFlash draft model. NaiveAI reports approximately 50 tokens per second per user in Standard mode and a peak single-stream decode rate of 2,122 tokens per second on eight GPUs in Ultrafast mode; the latter is the best one-second window across 41 HTML/SVG requests with thinking disabled and prefill excluded. On the same system, a speculative draft-verify-sample-commit round took 3.4 ms versus 12.3 ms with SGLang. The release provides a 49-shard BF16 checkpoint totaling approximately 617.8 GB, a 49-shard FP8 checkpoint totaling approximately 315.1 GB, and a 1.3 GB matching speculative-draft checkpoint. The repositories include custom Transformers implementations and a chat template. Although the launch describes NaiveRT as open source, the technical article says the optimized runtime repository contents and reproducibility materials are scheduled for October 12; the model and draft weights are already live. Planned API pricing is $0.10 per million input tokens, $0.40 per million output tokens, and $0.01 per million cache-read tokens.