arXiv:2602.14903cs.AI2026-02被引 3

分析大模型推理过程中的思维轨迹,发现关键步骤对正确答案的贡献机制。

The Potential of CoT for Reasoning: A Closer Look at Trace Dynamics

  • 通过引入'潜力'概念量化每步推理对答案正确性的提升
  • 发现推理轨迹常有非单调跳跃和意外正确猜测,部分逻辑难解释
  • 仅需20%强模型的推理步骤,弱模型即可解锁难题解法,具可迁移性

链式思维(CoT)提示已成为大语言模型(LLMs)生成类推理回答的标准方法,使模型在给出最终答案前展示中间步骤。尽管其与人类推理的相似性显著,但支撑其成功的核心机制仍不明确。本文深入分析来自竞赛级数学题的CoT推理轨迹,旨在理解哪些部分真正推动了最终答案的得出。为此,我们提出‘潜力’这一概念,用于量化特定推理步骤对正确完成答案的概率提升。分析发现:(1)潜力常呈现非单调性(源于偏离主线的思考);(2)出现急剧但难以解读的峰值(如关键洞察或跳跃式推理);(3)有时模型在无相关理由支持下偶然答对。尽管部分行为可解释且符合人类直觉(如洞察与离题),但某些模式仍难以从人类视角理解。进一步研究了CoT可迁移性:测量弱模型在接收强模型部分CoT时的表现。结果表明,仅20%的强模型推理路径即可‘解锁’弱模型此前无法解决的问题,凸显了CoT机制的高度可迁移性。

原文摘要 · Abstract (English)

Chain-of-thought (CoT) prompting is a de-facto standard technique to elicit reasoning-like responses from large language models (LLMs), allowing them to spell out individual steps before giving a final answer. While the resemblance to human-like reasoning is undeniable, the driving forces underpinning the success of CoT reasoning still remain largely unclear. In this work, we perform an in-depth analysis of CoT traces originating from competition-level mathematics questions, with the aim of better understanding how, and which parts of CoT actually contribute to the final answer. To this end, we introduce the notion of a potential, quantifying how much a given part of CoT increases the likelihood of a correct completion. Upon examination of reasoning traces through the lens of the potential, we identify surprising patterns including (1) its often strong non-monotonicity (due to reasoning tangents), (2) very sharp but sometimes tough to interpret spikes (reasoning insights and jumps) as well as (3) at times lucky guesses, where the model arrives at the correct answer without providing any relevant justifications before. While some of the behaviours of the potential are readily interpretable and align with human intuition (such as insights and tangents), others remain difficult to understand from a human perspective. To further quantify the reliance of LLMs on reasoning insights, we investigate the notion of CoT transferability, where we measure the potential of a weaker model under the partial CoT from another, stronger model. Indeed aligning with our previous results, we find that as little as 20% of partial CoT can ``unlock'' the performance of the weaker model on problems that were previously unsolvable for it, highlighting that a large part of the mechanics underpinning CoT are transferable.

链式思维推理分析模型可迁移性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。