1.71B and 4.11B (dense; marketed as 1.7B and 4B)
1,048,576 tokens
Apache 2.0
Same-day release of two compact, general-purpose dense language models designed for local and on-device conversation, writing, translation, reasoning, coding, tool use, and agentic workflows. Both models use a hybrid attention architecture that repeats three 512-token sliding-window attention layers followed by one full-attention layer, allowing them to natively support a 1,048,576-token context window while reducing KV-cache and compute requirements relative to full attention at every layer. Spark-X2.5-1.7B contains 1.71B parameters, 28 layers, a 2,048-dimensional hidden state, eight attention heads, and two key/value heads; Spark-X2.5-4B contains 4.11B parameters, 36 layers, a 2,560-dimensional hidden state, 16 attention heads, and four key/value heads. Both use 256-dimensional attention heads, partial rotary embeddings, grouped-query attention, headwise attention-output gates, tied embeddings, a 131,072-token vocabulary, and BF16 weights. The family was pretrained on approximately 20 trillion tokens drawn from web pages, books, academic publications, code, and encyclopedic material, followed by a dedicated long-context stage comprising hundreds of billions of tokens at sequence lengths extending to 1M. Post-training combines supervised fine-tuning with large-scale reinforcement learning across language understanding, reasoning, programming, instruction following, and tool-augmented agent behavior; specialized teacher policies are consolidated into the deployable models through MOPD. Thinking is enabled by default but can be disabled through the chat template, and the models support more than 200 languages. SparkLLM provides integrations or deployment guidance for SGLang, vLLM, llama.cpp, MLX, Ollama, LM Studio, LLaMA-Factory, Codex, Claude Code, OpenClaw, Hermes, and DeepSeek Harness. In vendor-reported thinking-mode evaluations, the 4B model scores 65.1 on BFCL-V4, 75.1 on tau2-bench, 30.4 on tau3-bench, 54.6 on MCP-Atlas, 40.9 on BrowseComp, 44.4 on SWE-Bench Pro, 53.3 on SWE-Bench Multilingual, 90.7 on AIME 2026, and 75.0 on IFBench. The 1.7B model scores 46.9 on BFCL-V4, 65.3 on tau2-bench, 29.7 on BrowseComp, 28.3 on SWE-Bench Verified, 69.4 on AIME 2026, and 66.3 on IFBench. Matching Base and GGUF variants are also available through the official release collection.