发现大模型推理跳步泛化失败根源,提出动态修正机制提升性能
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
- 定位错误集中于特定关键位置的注意力头,揭示内部竞争机制
- 移除错误处理头可恢复正确推理,证明其主导作用
- 提出测试时动态关闭错误头的方法,适配多种模型与任务
链式思维(CoT)推理已成为大语言模型解决复杂问题的标准范式。然而,近期研究发现,当所需推理步骤数超过训练分布时,模型性能急剧下降,而底层算法未变。这种推理跳步泛化失败的内在机制尚不明确。本文在多个领域任务上开展系统研究,发现错误集中出现在少数关键标记位置,而非均匀分布。深入分析表明,这些标记级错误源于内部竞争机制:部分注意力头(称作错误处理头,ep heads)会放大错误推理路径并抑制正确路径。值得注意的是,推理时单独移除某个ep头常能恢复正确预测。基于此,我们提出测试时推理修正方法,通过动态识别并禁用ep头来干预推理过程。在多种任务和大模型上的大量实验表明,该方法持续提升推理跳步泛化能力,凸显其有效性和潜力。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning has become the standard paradigm for enabling Large Language Models (LLMs) to solve complex problems. However, recent studies reveal a sharp performance drop in reasoning hop generalization scenarios, where the required number of reasoning steps exceeds training distributions while the underlying algorithm remains unchanged. The internal mechanisms driving this failure remain poorly understood. In this work, we conduct a systematic study on tasks from multiple domains, and find that errors concentrate at token positions of a few critical error types, rather than being uniformly distributed. Closer inspection reveals that these token-level erroneous predictions stem from internal competition mechanisms: certain attention heads, termed erroneous processing heads (ep heads), tip the balance by amplifying incorrect reasoning trajectories while suppressing correct ones. Notably, removing individual ep heads during inference can often restore the correct predictions. Motivated by these insights, we propose test-time correction of reasoning, a lightweight intervention method that dynamically identifies and deactivates ep heads in the reasoning process. Extensive experiments across different tasks and LLMs show that it consistently improves reasoning hop generalization, highlighting both its effectiveness and potential.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。