75.3B total / 9.3B active (hybrid MoE)
1M tokens
OpenMDW-1.1
Official open-weight deployment-optimized Nemotron release from NVIDIA that compresses Nemotron-3-Super-120B-A12B into a smaller hybrid Mamba-Transformer LatentMoE model tuned for interactive serving, reasoning-heavy generation, and long-context agentic workloads. The official model card and paper describe a hybrid architecture with interleaved Mamba, MoE, and attention layers plus Multi-Token Prediction, reduced from the parent model’s 120.7B total / 12.8B active parameters to 75.3B total / 9.3B active through the Iterative Puzzle compression pipeline. NVIDIA says the post-training recipe combines heterogeneous MoE channel pruning, heterogeneous active-expert reduction, Mamba SSM state pruning, long-context knowledge distillation recovery, reinforcement learning focused on software-engineering and agentic skills, and deployment-oriented FP8 and NVFP4 quantization. Release materials claim about 2x higher server throughput than Nemotron-3-Super on a single 8xB200 node at matched user-throughput constraints and an increase from 1 to 8 sustainable concurrent 1M-token requests on a single H100 GPU, while preserving strong performance across reasoning, coding, multilingual, long-context, and agentic benchmarks including MMLU-Pro, AIME25, GPQA, LiveCodeBench, SWE-Bench, Terminal Bench hard subset, TauBench V2, RULER, and WMT24++. NVIDIA positions the model as commercially usable and optimized for collaborative agents, chat systems, RAG applications, and other high-volume production workloads, with published support for Transformers and vLLM on Hopper and Blackwell hardware.