推理会损害大模型的归纳能力，需结构化干预才能提升
Reasoning Can Hurt the Inductive Abilities of Large Language Models
- 用棋类等游戏测试模型归纳规则能力，发现推理反而降低表现
- 在四类任务中，带思维链的模型表现普遍差于不推理的版本
- 提出三种错误模式并设计干预方法，无需重训练即可改进
大型语言模型在多个领域进展显著，但在从少量示例中推断潜在规则的归纳推理能力仍有限。通常认为思维链（CoT）提示能增强推理能力。我们通过构建四类受控诊断性游戏任务——国际象棋、德州扑克、骰子游戏和二十一点——来检验这一假设，这些任务均设有隐藏的人工定义规则。实验发现，使用思维链的推理模型在归纳任务中表现更差，常常不如不进行推理的模型。为此，我们提出了一个理论框架，揭示了推理步骤可能通过三种失败模式放大错误：子任务分解错误、子任务求解错误以及最终答案总结错误。基于理论与实证分析，我们设计了针对不同失败类型的结构化干预策略，可提升归纳准确率且无需重新训练。研究结果表明，有效的思维链推理不仅依赖更多步骤，更取决于步骤的结构合理性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable progress across domains, yet their ability to perform inductive reasoning - inferring latent rules from sparse examples - remains limited. It is often assumed that chain-of-thought (CoT) prompting, as used in Large Reasoning Models (LRMs), enhances such reasoning. We investigate this assumption with creating four controlled, diagnostic game-based tasks - chess, Texas Hold'em, dice games, and blackjack - with hidden human-defined rules. We find that CoT reasoning can degrade inductive performance, with LRMs often underperforming their non-reasoning counterparts. To explain this, we present a theoretical framework that reveals how reasoning steps can amplify error through three failure modes: incorrect sub-task decomposition, incorrect sub-task solving, and incorrect final answer summarization. Based on our theoretical and empirical analysis, we introduce structured interventions that adapt CoT generation according to our identified failure types. These interventions improve inductive accuracy without retraining. Our findings suggest that effective (CoT) reasoning depends not only on taking more steps but also on ensuring those steps are well-structured.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。