mini: 16B nominal non-embedding / 1.4B active (16,255,643,392 stored); flash: 100B nominal non-embedding / 6.1B active (102,889,705,216 stored); MoE
32,768 tokens (both variants)
Apache 2.0 (both checkpoints; retain applicable MIT notices from Ling ancestry)
Instruction-tuned, text-only block-diffusion MoE models converted from the Ling-mini-2.0 and Ling-flash-2.0 autoregressive model family, rather than trained from scratch like the original LLaDA. Tokens are iteratively unmasked in parallel with bidirectional attention inside each block; blocks are generated sequentially, conditioned on preceding blocks. The dedicated LLaDA2.0 report describes continual pretraining with a Warmup-Stable-Decay block-size curriculum: increase from 1 through 4, 32 and 64 to full-sequence 4,096-token diffusion, train at that scale, then reduce to compact blocks such as 32. Document-level attention prevents cross-document leakage, and top-k checkpoint averaging consolidates the converted model. Post-training uses block-diffusion SFT with bounded masking rates and complementary masks, followed by diffusion-ELBO-based DPO on 1.5 million preference pairs; reinforcement-learning enhancement was announced as future work. Mini has 20 layers and flash 32; both use 256 routed experts with eight selected per token plus one shared expert, RoPE and a 157,184-token vocabulary. The release emphasizes coding, mathematical reasoning and tool calling; official model cards report HumanEval 86.59/94.51 and AIME 2025 36.67/60.00 for mini/flash. Recommended decoding uses temperature 0, 32-token blocks and up to 32 refinement steps per block, with confidence threshold 0.95. Native context is 32K; the report tests 64K with inference-time YaRN scaling at reduced accuracy, not a native 64K release. Optimized block decoding can reuse KV cache in dInfer/SGLang; speed depends on decoder and serving setup. Earlier preview checkpoints and later confidence-aware-parallel (CAP) checkpoints are separate releases and are not included here.