CoT推理中的隐藏状态藏着解题关键,能修复错误推理链。
When Chain-of-Thought Fails, the Solution Hides in the Hidden States

- 通过替换隐藏状态,从正确推理中提取信息修复错误链
- 修复后准确率高于原完整CoT和直接提示,最高提升12.3%
- 中间层和后期推理词蕴含核心解题信息,适合模型可解释性研究
Chain-of-Thought(CoT)推理是否具有计算价值,取决于其中的中间步骤是否包含任务相关的信息。本研究针对GSM8K数据集,采用激活修补技术:将同一问题在生成CoT时的逐标记隐藏状态,转移到直接回答的运行中,测量对最终答案准确率的影响。结果显示,修补后生成的准确率显著高于直接提示和原始CoT轨迹,说明单个CoT标记即使在错误推理链中仍可能编码足够信息以恢复正确答案。该任务相关信息在正确推理中更丰富,且集中在中后期层与推理序列早期出现。语言类标记(如动词、实体)携带解题导向信息,引导生成走向正确路径;而数学符号标记则编码答案邻近内容,但成功率低。修补后的输出通常更短,却超越完整CoT轨迹的准确率,表明完整的推理链并非必需。这些发现表明,CoT中存在可恢复的、标记级别的求解信息,为推理的表征机制与失效原因提供了新见解。
原文摘要 · Abstract (English)
Whether intermediate reasoning is computationally useful or merely explanatory depends on whether chain-of-thought (CoT) tokens contain task-relevant information. We present a mechanistic causal analysis of CoT on GSM8K using activation patching: transferring token-level hidden states from a CoT generation to a direct-answer run for the same question, then measuring the effect on final-answer accuracy. Across models, generating after patching yields substantially higher accuracy than both direct-answer prompting and the original CoT trace, revealing that individual CoT tokens can encode sufficient information to recover the correct answer, even when the original trace is incorrect. This task-relevant information is more prevalent in correct than incorrect CoT runs and is unevenly distributed across tokens, concentrating in mid-to-late layers and appearing earlier in the reasoning trace. Moreover, patching language tokens such as verbs and entities carry task-solving information that steers generation toward correct reasoning, whereas mathematical tokens encode answer-proximal content that rarely succeeds. Patched outputs are often shorter and yet exceed the accuracy of a full CoT trace, suggesting complete reasoning chains are not always necessary. Together, these findings demonstrate that CoT encodes recoverable, token-level problem-solving information, offering new insight into how reasoning is represented and where it breaks down.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。