发现大模型隐式推理依赖捷径学习，泛化能力差
Implicit Reasoning in Transformers is Reasoning through Shortcuts
- 用固定模式数据训练，模型可实现高效隐式多步推理
- 在相似模式任务上准确率高，但跨域泛化能力弱
- 揭示大模型隐式推理本质是捷径学习，适合特定场景
测试时计算正成为提升语言模型复杂多步推理能力的新范式，如OpenAI的o1、o3及DeepSeek的R1所示。相比显式推理，隐式推理更具推理效率，生成令牌更少。然而，为何高级推理能力在隐式推理中难以显现？本文从头训练GPT-2于精选多步数学推理数据集，并开展分析实验，探究语言模型在多步任务中如何进行隐式推理。研究发现：1）模型可通过隐式推理实现逐步推理，在域内和域外测试中均取得高准确率，但仅当训练数据具有固定模式时才有效；2）若训练数据无固定模式，隐式推理能力易过拟合特定模式，无法进一步泛化。该现象亦见于当前先进大模型。结果表明，语言模型通过捷径学习获得隐式推理能力，虽在相似模式任务中表现优异，却缺乏通用性。
原文摘要 · Abstract (English)
Test-time compute is emerging as a new paradigm for enhancing language models' complex multi-step reasoning capabilities, as demonstrated by the success of OpenAI's o1 and o3, as well as DeepSeek's R1. Compared to explicit reasoning in test-time compute, implicit reasoning is more inference-efficient, requiring fewer generated tokens. However, why does the advanced reasoning capability fail to emerge in the implicit reasoning style? In this work, we train GPT-2 from scratch on a curated multi-step mathematical reasoning dataset and conduct analytical experiments to investigate how language models perform implicit reasoning in multi-step tasks. Our findings reveal: 1) Language models can perform step-by-step reasoning and achieve high accuracy in both in-domain and out-of-domain tests via implicit reasoning. However, this capability only emerges when trained on fixed-pattern data. 2) Conversely, implicit reasoning abilities emerging from training on unfixed-pattern data tend to overfit a specific pattern and fail to generalize further. Notably, this limitation is also observed in state-of-the-art large language models. These findings suggest that language models acquire implicit reasoning through shortcut learning, enabling strong performance on tasks with similar patterns while lacking generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。