让大模型在隐藏状态中完成推理,提升效率与可解释性。
A Survey on Latent Reasoning
- 通过隐藏状态而非文本逐步推理解题,突破语言表达限制。
- 提出无限深度隐式推理框架,实现全局一致且可逆的推导过程。
- 适合关注大模型认知机制、高效推理方法的研究者参考。
大语言模型(LLMs)在显式思维链(CoT)引导下展现出卓越的推理能力,但其依赖自然语言表达中间步骤,限制了模型的表达带宽。隐式推理通过在模型连续隐藏状态中完成多步推断,无需逐标记监督,解决了这一瓶颈。本文综述了隐式推理领域的最新进展:首先分析神经网络层作为推理计算基底的作用,揭示分层表征如何支持复杂变换;接着探讨多种隐式推理方法,包括基于激活的递归、隐藏状态传播及压缩或内化显式推理轨迹的微调策略;最后介绍先进范式,如基于掩码扩散模型实现的无限深度隐式推理,支持全局一致且可逆的推理过程。通过整合这些视角,本文旨在厘清隐式推理的概念图景,并指明大模型认知前沿的研究方向。相关论文与代码资源详见 GitHub 仓库:https://github.com/multimodal-art-projection/LatentCoT-Horizon/。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: https://github.com/multimodal-art-projection/LatentCoT-Horizon/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。