16B (dense)
Apache 2.0
Diffusion language model with token-editing capability allowing self-correction during generation; generates tokens in parallel and refines through diffusion-style process rather than strict autoregressive generation; two operational modes: Speedy Mode (S Mode) for speed with T2T refinement, Quality Mode (Q Mode) for superior benchmark performance; peak inference speed of 1586.93 TPS after quantization; 32K context window; 20 layers, 16 attention heads, 157K vocabulary; strong benchmarks: MMLU-Pro, C-EVAL (knowledge), BIG-Bench Hard, DROP (reasoning), HumanEval+, Spider (coding), CMATH, GSM-Plus (math); overall average score 63.90; requires ~48GB VRAM