120B total / 12B active (multimodal MoE)
1,048,576 tokens (1M)
OpenMDW License Agreement, version 1.1
Open-weight vision-language reasoning and agent model combining interleaved Mamba-2, sparse Mixture-of-Experts, and attention layers with a C-RADIO vision encoder and vision-language connector. It accepts text, images, and video and produces text, supporting reasoning, coding, tool use, document understanding, visual grounding, OCR, and video understanding. The model has 512 routed experts per MoE layer, activates 12B of its 120B parameters per token, and includes a shared Multi-Token Prediction head for native speculative decoding. Its chat template supports switchable thinking and non-thinking operation. NVIDIA extended the language backbone through software-engineering mid-training, 1M-context training, multimodal training, and post-training using reasoning and agent trajectories; the release also links its pretraining and post-training datasets and reproducible NeMo Gym evaluation recipes. Official BF16 reference weights target customization, domain adaptation, and full-precision evaluation and can be deployed on four H100s or two GB200s, while the companion NVFP4 checkpoint targets more efficient serving. NVIDIA reports 89.52 on GPQA Diamond, 88.22 on LiveCodeBench v6, 71.10 on SWE-bench Verified, 51.78 on SWE-bench Pro, 66.70 on Terminal-Bench 2.1, 68.12 on English OCRBench v2, 80.82 on CharXiv reasoning, and 51.14 on OSWorld-Verified.