305B total (MoE)
1,048,576 tokens
MIT
Experimental open-weight multimodal continuation of DeepSeek-V4-Flash, released after an August 21 API preview and produced by adding visual modules to the Flash architecture followed by continued training. The official checkpoint contains approximately 305B total parameters across 48 mixed-precision safetensors shards totaling about 168 GB. Its 43-layer sparse MoE language backbone uses a 4,096-dimensional hidden state, 256 routed experts plus one shared expert, six routed experts selected per token, 64 attention heads, one key/value head, DFlash sparse attention with an index top-k of 512 and a 128-token sliding window, Hyper-Connections, and three DSpark next-token-prediction layers for integrated speculative decoding. YaRN scales the original 65,536-token window to 1,048,576 tokens. The added vision tower has 32 layers, a 1,024-dimensional state, 16 attention heads, 14-pixel patches, a downsampling ratio of 3, and a maximum of 384 visual tokens. The repository combines FP4 expert weights with dynamic blockwise FP8 quantization for supported components, provides OpenAI-style multimodal message encoding plus `<image>path</image>` notation, and includes a minimal PyTorch implementation covering the vision encoder and aligner, DFlash, MoE, Hyper-Connections, and DSpark, alongside SGLang deployment guidance. DeepSeek reports text-agent scores of 83.9 on Terminal-Bench 2.1, 59.3 on DeepSWE, 75.9 on Toolathlon-Verified, and 75.3 on CyberGym, plus multimodal scores of 36.5 on ApexBench pass@1, 27.3 on Agents' Last Exam, 64.3 on Chartography, and 35.0 on ZeroBench pass@5. DeepSeek positions it as retaining text-agent performance comparable to DeepSeek-V4-Flash-0731 while substantially improving multimodal-agent capabilities.