600B total / 27B active (MoE)
1,048,576 tokens
StepFun Model License 1.0 (current official distribution; September 20 leaked snapshot carried Apache-2.0 metadata)
Official open-weight preview of StepFun's flagship model for software engineering, long-horizon agents, professional knowledge work, and finance. Step 5 Preview is a native multimodal sparse Mixture-of-Experts model marketed as 600B total and 27B activated parameters; Hugging Face inventories approximately 604.34B serialized parameters. Its narrow-and-deep language backbone has 92 transformer blocks, a 4,096-dimensional hidden state, 64 attention heads in four groups, 352 routed experts selecting eight per token, three initial dense layers, a 128,896-token vocabulary, and three attached multi-token-prediction layers. Attention follows a repeating 3:1 pattern of 512-token Gated Sliding Window Attention and full-attention layers using StepFun's Sparse GQA, which combines sparse indexing with block-wise token merging for million-token operation. The released configuration natively specifies 1,048,576 positions. A 47-layer, 1,536-wide vision encoder with 16 heads and 14-by-14 patches provides visual input; StepFun describes text, image, and video support, although the concise GitHub specification lists text and image input while the hosted API documentation additionally documents video. The official release includes a blockwise FP8 E4M3 checkpoint with dynamic activation quantization and a BF16 checkpoint. Each contains 24 main language-model shards plus vision and MTP artifacts; all safetensors total approximately 650.3 GB for FP8 and 1.215 TB for BF16. StepFun's technical report describes mixed-FP8 training with FP32 hidden states, a role-based agentic data-synthesis system, bitwise training-inference alignment across tensor-parallel sizes, RL trace replay for routed experts and top-p candidate sets, context compaction trained end to end, and token- and turn-level credit assignment for very long trajectories. StepFun reports an end-to-end rollout sample-drop rate below 1%. At High reasoning effort, reported results include 67.7 on DeepSWE v1.1, 80.5 on ProgramBench, 49.0 on StepCodeBench, 33.3 on Terminal-Bench v4, 29.5 on Agents' Last Exam CLI, 66.4 on FrontierFinance, and 83.3 on DRACO. StepFun's official September 20 FP8 repository history labeled the accidentally exposed snapshot `apache-2.0`, although it included no license text. A surviving BF16 fork contains safetensors byte-identical to the October 15 official release. StepFun subsequently replaced the Apache metadata with StepFun Model License 1.0. Whether the original metadata created an enforceable Apache grant for leaked copies remains unresolved. Official deployment paths cover vLLM, SGLang, and standalone Transformers on B300 and H200 systems with tensor, data, and expert parallelism plus EAGLE speculative decoding. The hosted API supports low, medium, and high reasoning, tool calling, prompt caching, JSON Mode, JSON Schema, and a documented 64K maximum output; the GitHub model specification separately claims output up to 1M tokens.