~63.2B total shipped parameters across two towers (about 30B AR/context tower + 30B diffusion/denoiser tower); ~3B active per tower
128K tokens
NVIDIA Nemotron Open Model License
Official open-weight diffusion language model release from NVIDIA that turns the open Nemotron-3-Nano-30B-A3B backbone into a two-tower block-wise autoregressive diffusion generator. The architecture splits generation into a frozen autoregressive context tower and a trainable diffusion denoiser tower, with both towers copied from the same 52-layer hybrid Mamba-2, attention, and MoE backbone. NVIDIA says the denoiser tower adds bidirectional in-block attention, layer-aligned cross-attention to the context tower, context-seeded Mamba state initialization, and timestep conditioning, while keeping the AR tower frozen. The model card says the backbone was pretrained on about 25T tokens, the denoiser adaptation was trained on about 2.1T tokens, and the released checkpoint ships both towers together in BF16. Official benchmarks position the model as preserving 98.7% of the aggregate quality of its autoregressive Nemotron-3-Nano-30B-A3B baseline while delivering 2.42x wall-clock generation throughput at the default operating point, with published comparisons across MMLU, ARC-Challenge, HumanEval, MBPP, GSM8K, MATH-500, and multilingual tasks. NVIDIA also highlights commercial usability, support for Transformers, vLLM, SGLang, and Docker workflows, and a decoding design that generates token blocks iteratively in parallel rather than strictly one token at a time.