将知识与推理分离，实现更高效的记忆与思考。
Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning
- 知识与推理模块解耦，用记忆层存知识，多个推理器迭代调用。
- 7B模型仅用62.6%数据即达基线效果，35B模型推理快4倍。
- 适合需要高效推理和小样本训练的场景。
我们提出Mobius-v0架构，包含全局共享的记忆（前馈网络）存储知识向量，以及多个自注意力推理器，通过迭代方式完成组合推理。推理器利用隐藏状态作为缓存和载体，反复查询记忆中的知识向量，并将知识回传至推理模块。该知识-推理解耦结构提升了知识压缩效率与推理性能。基于此架构：1）我们从零训练的7B模型仅需基线62.6%的训练数据，即可达到相近下游表现；2）从Qwen3.5-35B持续预训练的Intern-S2-Mobius，在保持相近下游性能的同时，实现近4倍的端到端推理加速。
原文摘要 · Abstract (English)
We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve compositional reasoning. Using hidden states as cache and carrier, reasoners repeatedly query memory for required knowledge-vectors, while the knowledge is transmitted back to reasoning operators. Through this knowledge-reasoning-separation architecture, Mobius achieves better knowledge compression and reasoning efficiency. Built upon Mobius-v0 architecture: 1) Our 7B model trained-from-scratch achieves similar downstream score as a 7B Transformer baseline with 62.6% of baseline's training data. 2) Our Intern-S2-Mobius, continually-pretrained from Qwen3.5-35B, achieves similar downstream score while delivering nearly 4x end-to-end inference speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。