arXiv:2603.01437cs.AI2026-03被引 11

发现大模型先确定答案再编理由,可能导致错误推理。

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

  • 模型在生成思维链前已决定答案,可通过激活值预测结果
  • 沿探测方向调整激活值可显著改变答案,准确率超0.9 AUC
  • 错误推理分虚构前提和结论脱节两类,适合关注模型可信性者阅读

随着思维链(CoT)成为提升大语言模型推理能力的核心方法,其也展现出解释性的潜力,即通过模型自述的推理过程理解其决策。然而,这种解释性依赖于思维链的真实性——即所述推理是否反映真实决策过程。我们提供了机制证据表明,指令微调模型通常在生成思维链之前就已确定答案。在思维链最后一个词之前的残差流激活上训练线性探测器,可在多数任务中以超过0.9的AUC预测最终答案。这些方向不仅具有预测性,还具因果性:沿探测方向引导激活值常导致答案翻转,翻转率远高于范数匹配的正交基线。当引导产生错误答案时,观察到两种不同失败模式:虚构(编造虚假前提)与非蕴含(陈述正确前提但得出无支持结论)。尽管后验推理在模型已有正确预判时可能有用,但上述失败模式表明,从错误信念出发的推理可能导致不良行为。

原文摘要 · Abstract (English)

As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning. However, the utility of CoT toward interpretability depends upon its faithfulness---whether the model's stated reasoning reflects the underlying decision process. We provide mechanistic evidence that instruction-tuned models often determine their answer before generating CoT. Training linear probes on residual stream activations at the last token before CoT, we can predict the model's final answer with >0.9 AUC on most tasks. We find that these directions are not only predictive, but also causal: steering activations along the probe direction often flips model answers, with flip rates substantially exceeding norm-matched orthogonal baselines across most model-dataset pairs. When steering induces incorrect answers, we observe two distinct failure modes: confabulation (fabricating false premises) and non-entailment (stating correct premises but drawing unsupported conclusions). While post-hoc reasoning may be instrumentally useful when the model has a correct pre-CoT belief, these failure modes suggest it can result in undesirable behaviors when reasoning from a false belief.

思维链模型解释因果推理大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。