770B total / 49B active MoE backbone + 10B total / 0.7B active MTP layer
1,048,576 tokens
Apache 2.0
Open-weight early preview of Tencent's fourth-generation Hy flagship, released as paired BF16 and FP8 instruction-tuned checkpoints. The backbone contains 770 billion total parameters with 49 billion activated per token, alongside one native Multi-Token Prediction layer containing 10B total and 0.7B active parameters for speculative decoding. Its 78-layer architecture uses one dense feed-forward layer followed by 77 Mixture-of-Experts layers, each containing 256 routed experts and one shared expert; every token activates eight routed experts plus the shared expert. The attention system uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse-index reuse, while identity Hyper-Connections expand the residual pathway to four streams. Additional configuration details include a 6,144-dimensional hidden state, 64 attention heads, 2,048-dimensional query compression, 512-dimensional key-value compression, a sparse-attention index top-k of 2,048, and a native 1,048,576-token context window. Tencent positions Hy4 preview as a productivity-oriented agent model co-designed with software engineers, game developers, financial analysts, security specialists, CodeBuddy, and WorkBuddy, emphasizing long-horizon software development, frontend interaction quality, cross-file office analysis, financial modeling, playable game-prototype generation, and scientific research. In a blind internal evaluation, 163 Tencent experts rated outputs on 203 engineering tasks; Tencent reports a 2.99 out of 4 average for Hy4 preview versus 2.92 for GLM-5.3 and 2.94 for Kimi K3, with 46.8% wins, 12.8% ties, and 40.4% losses against GLM-5.3 and 51.2% wins, 7.9% ties, and 40.9% losses against Kimi K3. Published maximum-effort evaluations report 82.9 on SWE-bench Multilingual, 65.7 on SWE-bench Pro, 64.3 on DeepSWE, 85.4 on Terminal-Bench 2.1, 78.4 on CyberGym, 74.1 on Toolathlon-Verified, 65.4 on OneMillionBench with tools, 83.7 on MCP-Atlas, 92.3 on GPQA Diamond, 74.2 on MathArena Apex 2025, and 55.4 on Humanity's Last Exam with tools. Reasoning defaults to high effort, while a no_think mode is available for direct responses. Tencent explicitly identifies excessive reasoning time on difficult tasks and a tendency to over-verify its work as known preview limitations. The official Hugging Face repositories contain approximately 1.56 TB of BF16 files across 131 safetensors shards and 814 GB of FP8 files across 130 shards. Official deployment recipes support vLLM and SGLang, MTP speculative decoding, automatic tool selection, and dedicated Hy4 reasoning and tool-call parsers.