280B total / 14B active per token, plus 0.36B ECHO parameters
131,072 tokens during pretraining; up to 1,048,576 tokens for fine-tuning and inference
ERNIE 5.5 Flash Base Model License Agreement v1.0 (custom; licensed territory excludes the European Union, United Kingdom, and South Korea; commercial use at or below US$50 million in related annual revenue requires a one-time notice and prominent "Powered by ERNIE" attribution, while commercial use above that threshold requires prior written authorization; additional derivative-model and acceptable-use restrictions apply)
Open-weight, text-only sparse mixture-of-experts base model released at the end of pretraining, before any mid-training, instruction tuning, reinforcement learning, or behavioral post-training. ERNIE-5.5-Flash-Base was pretrained from scratch on 33.5 trillion tokens with the Muon optimizer and is intended as a foundation for domain adaptation, instruction tuning, and reinforcement-learning research rather than a ready-made chat assistant. Its 43-layer backbone contains one dense feed-forward layer followed by 42 Multi-Router Latent MoE layers. Each MoE layer has 512 routed experts with ten selected per token, one shared expert, a 2,048-dimensional latent dispatch space, and auxiliary-loss-free load balancing. The hybrid attention stack combines 37 Heavily Compressed Attention layers, which cache one entry per 128 tokens and use 512-dimensional heads, with six full-context Multi-head Latent Attention layers using DeepSeek Sparse Attention and 256-dimensional heads. Every attention layer also uses QK normalization, Low-Rank Gated Attention, and Virtual Head Attention. Other architectural features include manifold-constrained Hyper-Connections with expansion rate four, a MultiMax vocabulary head with learned piecewise temperature, and an approximately 200K-token BPE vocabulary that groups numeric digits right-to-left. ECHO reuses the backbone's final layer for multi-token prediction and as an integrated speculative-decoding draft model instead of attaching a separate MTP module. Pretraining data emphasizes multilingual web text, repository-scale code, software-development trajectories such as commits, issues, pull requests, reviews and CI-verified fixes, and natural plus synthetic long-context tasks. Baidu reports publisher-run base-model results under a unified evaluation pipeline including 93.3 MMLU, 78.4 MMLU-Pro, 60.2 GPQA, 82.3 HumanEval+, 81.5 MBPP+, 40.1 LiveCodeBench, 50.3 RepoBench, 91.7 RepoQA, and 48.7 LongBench-v2. These are base-model few-shot or direct-completion evaluations and should not be compared directly with instruction-model or agent-harness results. Official serving support is provided through SGLang and Baidu's vLLM-ERNIE integration. The released FP8 checkpoint contains approximately 286.2B serialized parameters and occupies about 304 GB, while the BF16 repository occupies about 573 GB. Both are provided as sharded safetensors through Hugging Face and ModelScope.