揭秘大模型多步推理的内在机制，打破黑箱迷雾。
Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models
- 从隐藏激活到显式语言推理，系统梳理机制框架
- 提出七个核心研究问题，构建理解多步推理的理论体系
- 适合关注模型可解释性与推理机理的研究者
大型语言模型（LLMs）展现出解决需要多步推理问题的惊人能力，但其内部实现这些能力的机制仍不明确。与以往主要关注提升性能的工程方法的综述不同，本文全面回顾了支撑LLM多步推理的内在机制。我们围绕一个包含七个相互关联研究问题的概念框架展开，涵盖模型如何在隐藏激活中执行隐式多跳推理，以及显式语言推理如何重塑内部计算过程。最后，我们指出了五个未来机制研究的方向。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable abilities to solve problems requiring multiple reasoning steps, yet the internal mechanisms enabling such capabilities remain elusive. Unlike existing surveys that primarily focus on engineering methods to enhance performance, this survey provides a comprehensive overview of the mechanisms underlying LLM multi-step reasoning. We organize the survey around a conceptual framework comprising seven interconnected research questions, from how LLMs execute implicit multi-hop reasoning within hidden activations to how verbalized explicit reasoning remodels the internal computation. Finally, we highlight five research directions for future mechanistic studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。