30B total, approximately 3B active
262,144 tokens
Apache License 2.0
Open-weight mixture-of-experts model purpose-trained as a long-term-memory writer for LLM applications. Hy-Memory-30B-A3B replaces the general-purpose language model used along the write path of Mem0, MemOS, and Hy-Memory while leaving each framework's schemas, retrieval logic, reader model, and memory-bank implementation intact. It decides what information should persist, preserves exact names, numbers, dates, paths, preferences, and state transitions, reconciles new evidence with existing memories, and emits the structured operations expected by each framework. The Hy3 architecture contains 48 Transformer layers with a 2,048-dimensional hidden state, 32 query heads, 4 key-value heads, 128 routed experts, 8 selected experts per token, one shared expert, a 120,832-token vocabulary, and a configured 262,144-token context window. It starts from an instruction-tuned Hy3-30B-A3B checkpoint and remains capable of ordinary multi-turn chat and text generation. Its official chat template supports system, user, assistant, and tool roles, structured tool calls and responses, and high, low, or no-think reasoning modes; however, Tencent trains and evaluates this release primarily as a specialized memory writer rather than as a broadly benchmarked general assistant. Training uses two stages. Memory On-Policy Distillation first teaches the student to operate five real deployment configurations spanning Mem0, MemOS, and Hy-Memory Chat, Working, and Coding. A frozen Hy3 flagship teacher supplies top-50 reverse-KL supervision on student-generated token sequences and memory states rather than separate ideal trajectories. Attributed Memory Reinforcement Learning then samples eight candidate trajectories for windows of up to five chunks, executes each candidate against an isolated memory bank, and carries the highest-scoring final bank into the next window. Its reward combines dense per-chunk recall probes, cross-chunk outcome questions attributed to the chunks that introduced their evidence, adversarial trap probes, and a bank-growth penalty that discourages storing everything. Optimization uses a PPO-style clipped objective with chunk-level GRPO advantages and replayed MoE routing, without a KL penalty. Tencent reports training the RL stage on 128 H20 GPUs with batches of 512 trajectories. The reward corpus covers 12,671 windows and 57,616 chunks with 164,576 positive recall probes, 8,599 traps, and approximately 60,800 attributed outcome questions. On fixed 20% benchmark subsamples with the reader, retriever, judge, prompts, and framework settings held constant, Tencent reports that Hy-Memory-30B-A3B ranks first across Mem0, Hy-Memory, MemOS, and the unseen EverMemOS framework, attaining a 55.49 macro average versus 54.62 for Claude Opus 5, 53.64 for DeepSeek V4.1 Flash, and 47.66 for the Hy3-A3B base checkpoint. The released BF16 checkpoint occupies approximately 60.1 GB and supports Transformers, vLLM, and SGLang.