35B-class sparse multimodal model; 35,964,405,616 parameters in the BF16 checkpoint
262,144 tokens
Apache-2.0
Official open-weight scientific multimodal foundation model built on InternLM’s experimental Mobius-v0 architecture and continued-pretrained from Qwen3.5-35B before supervised fine-tuning and reinforcement learning. Rather than binding feed-forward knowledge storage to individual Transformer layers, Mobius separates knowledge from reasoning computation through a globally shared Memory containing 2,560 small routed experts and multiple recurrent Reasoners that repeatedly query and refine hidden states against that shared repository. This design enables Backward Residual Connection, allowing reasoning stages to access knowledge outside their local forward layer hierarchy, and Dynamic Latent Reasoning, which performs iterative deliberation and refinement in continuous hidden states before decoding. The published configuration uses 40 language-model layers with a three-to-one mixture of linear-attention and full-attention layers, 16 attention heads, eight selected memory experts per token, a 2,048-dimensional hidden state, and a separate Multi-Token Prediction layer with 256 experts and eight selected per token. InternLM reports that Mobius matches or improves upon Qwen3.5-35B across aggregate general and scientific reasoning evaluations while producing substantially shorter visible reasoning traces, yielding nearly four times higher end-to-end inference efficiency in its reported evaluation. Scientific gains are highlighted on Biology-Instructions, Mol-Instructions, and MolecularIQ. The checkpoint supports text, image, video, scientific sequence, and time-series inputs; native thinking traces and structured tool calling are exposed through its chat template. BF16 and FP8 weights are released under Apache 2.0, with deployment guidance for Transformers, LMDeploy, vLLM, and SGLang and recommended MTP speculative decoding support.