揭秘大模型如何在不输出步骤的情况下完成复杂推理。
Implicit Reasoning in Large Language Models: A Comprehensive Survey
- 从显式思维链转向隐式推理,利用内部结构完成计算
- 提出三类执行范式:隐式优化、信号引导控制、层递归执行
- 适合关注模型内部工作机制的研究者与开发者
大型语言模型(LLMs)在多种任务中展现出强大的泛化能力。推理是解决多步问题和复杂决策的核心。为提升推理效率,近期研究从显式思维链提示转向隐式推理,即通过潜在结构在不生成中间文本步骤的情况下完成推理。隐式推理具有生成成本低、推理速度快、更契合内部计算等优势。尽管已有研究探讨过推理中的潜在表示,但针对模型内部推理机制的系统性分析仍属空白。本文填补该空白,提出以执行范式为中心的分类体系,聚焦于‘内部计算如何及在何处展开’。将现有方法归纳为三类:隐式优化、信号引导控制、层递归执行。同时综述支持隐式推理存在的结构、行为和表示证据,并系统梳理评估指标与基准测试。项目持续更新:https://github.com/digailab/awesome-llm-implicit-reasoning。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated strong generalization across a wide range of tasks. Reasoning with LLMs is central to solving multi-step problems and complex decision-making. To support efficient reasoning, recent studies have shifted attention from explicit chain-of-thought prompting toward implicit reasoning, where reasoning occurs silently via latent structures without emitting intermediate textual steps. Implicit reasoning brings advantages such as lower generation cost, faster inference, and better alignment with internal computation. Although prior surveys have discussed latent representations in the context of reasoning, a dedicated and mechanism-level examination of how reasoning unfolds internally within LLMs remains absent. This survey fills that gap by introducing a taxonomy centered on execution paradigms, shifting the focus from representational forms to computational strategies. We organize existing methods into three execution paradigms based on \textbf{\textit{how and where internal computation unfolds}}: latent optimization, signal-guided control, and layer-recurrent execution. We also review structural, behavioral and representation-based evidence that supports the presence of implicit reasoning in LLMs. We further provide a structured overview of the evaluation metrics and benchmarks used in existing works to assess the effectiveness and reliability of implicit reasoning. We maintain a continuously updated project at: https://github.com/digailab/awesome-llm-implicit-reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。