让大模型像人一样用隐式记忆推理，更省力还更好用。
Beyond Words: A Latent Memory Approach to Internal Reasoning in LLMs
- 用隐式记忆模块替代显式语言推理，提升内部思考效率。
- 在简单GPT上加记忆模块，训练损失降低35%至57%。
- 既支持高效推理，又可轻松添加可解释性输出通道。
大语言模型近年广泛采用思维链（CoT）范式，通过自然语言生成显式推理步骤，虽增强可解释性，但未必是内部推理的最优方式。人类认知依赖无需完全语言化的隐式记忆来回忆过往经验。本文提出将隐式记忆引入大模型内部推理的框架。初步实验表明，在基础GPT模型中加入隐式记忆模块（IMM），最终训练损失相比普通GPT基线降低35%至57%。该方法可轻松集成显式可解释性通道（如思维链解码器）。论文阐述了理论基础，提出可扩展记忆模块的技术机制，并讨论其在提升推理效率与鲁棒性方面的潜力，未来还可拓展为可审计的系统。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have popularized the chain-of-thought (CoT) paradigm, in which models produce explicit reasoning steps in natural language. Although this approach improves interpretability and facilitates external auditing, it may not represent the most computationally efficient method for internal reasoning. In contrast, human cognition relies on implicit mental representations that recall past sensory and episodic information without requiring complete verbalization. In this paper, we propose a framework that integrates implicit mental representations into the internal reasoning processes of LLMs. Preliminary experiments indicate that incorporating an Implicit Memory Module (IMM) into a simple GPT model yields a reduction of between 35% and 57% in final training loss compared to a regular GPT baseline. The addition of an explicit interpretability channel (e.g., a chain-of-thought decoder) is straightforward to implement within this approach. We outline theoretical foundations, propose technical mechanisms to scale the memory module, and discuss how these ideas may lead to more efficient and robust reasoning, with optional future extensions for explicit auditability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。