发现大模型推理中答案形成的关键转折点，可提前终止冗余步骤
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
- 通过早期退出法测量每步推理的因果重要性
- 多数任务在推理末期前1步即完成答案锁定，后续步骤无影响
- 可利用此特性提前终止推理，平均减少55%计算量
链式思维（CoT）是语言模型推理时扩展的主要范式，但各推理步骤对最终答案的因果影响尚不明确。我们通过早期退出法估计每一步的因果重要性，并研究多个模型家族推理轨迹中的答案形成过程。在多种任务中，推理通常会跨越一个‘承诺边界’——从临时中间猜测到稳定高置信度答案的突变。这一转变常在单步内完成，远早于推理块结束，其后出现‘附带性’的CoT步骤，对最终答案概率无影响。通过注意力探针，我们发现答案形成阶段可从中间推理步骤中高精度线性解码，且能泛化至未见任务。利用该信号，我们在承诺边界处提前退出推理块，平均将CoT长度减少55%，对模型性能影响微乎其微。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer poorly understood. We estimate each step's causal importance via early exit and use this measure to study how answers form across the reasoning traces of several model families. Across diverse tasks, we find that reasoning typically crosses a \emph{commitment boundary} -- a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by \emph{epiphenomenal} CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from intermediate reasoning steps with high accuracy and generalize robustly to unseen reasoning tasks. We exploit this signal to early-exit reasoning blocks at the commitment boundary, reducing the length of CoTs up to 55\% on average with negligible impact on model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。