arXiv:2505.17406cs.AI2025-05被引 10

发现大模型答对题却逻辑混乱,可能只是巧合。

Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoning

  • 用条件探针框架隔离推理过程,测试答案与理由的关联性。
  • 微小扰动下正确答案不变,但推理过程常变得不一致或荒谬。
  • 尤其在多步和常识任务中,答案与推理脱节现象严重。

尽管思维链(Chain-of-Thought, CoT)提示已成为大语言模型(LLMs)复杂推理的核心方法,其生成推理过程的真实性仍存疑问。本文检验了‘解耦假设’:即正确答案常掩盖脆弱、事后编造的解释,与模型预测无因果关联。为此,我们提出MATCHA——一种答案条件化探针框架。不同于传统仅关注最终输出准确率的评估方式,MATCHA通过以模型预测答案为条件生成内容,从而独立测试推理过程本身的稳定性。实验发现,当输入遭受不可察觉的微小扰动时,模型仍能维持正确答案,却生成不一致甚至荒谬的推理路径,即‘因错误原因而答对’。通过使用大模型作为评判者量化该鲁棒性差距,我们发现多步推理和常识任务比逻辑任务更易出现这种解耦现象。此外,我们证明由MATCHA生成的对抗样本可有效迁移至黑盒模型。这些发现揭示了CoT推理表面鲁棒性的幻象,强调未来架构必须强制实现答案与推理的真实一致性,而非仅追求表层准确率。

原文摘要 · Abstract (English)

While Chain-of-Thought (CoT) prompting has become a cornerstone for complex reasoning in Large Language Models (LLMs), the faithfulness of the generated reasoning remains an open question. We investigate the Decoupling Hypothesis: that correct answers often mask fragile, post-hoc rationalizations that are not causally tied to the model's prediction. To systematically verify this, we introduce MATCHA, a novel Answer-Conditioned Probing framework. Unlike standard evaluations that focus on final output accuracy, MATCHA isolates the reasoning phase by conditioning generation on the model's predicted answer, allowing us to stress-test the stability of the rationale itself. Our experiments reveal a critical vulnerability: under imperceptible input perturbations, LLMs frequently maintain the correct answer while generating inconsistent or nonsensical reasoning - effectively being ``Right for the Wrong Reasons''. Using LLM judges to quantify this robustness gap, we find that multi-step and commonsense tasks are significantly more susceptible to this decoupling than logical tasks. Furthermore, we demonstrate that adversarial examples generated by MATCHA transfer non-trivially to black-box models. Our findings expose the illusion of CoT robustness and underscore the need for future architectures that enforce genuine answer-reasoning consistency rather than mere surface-level accuracy.

大模型推理验证逻辑脆弱性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。