发现大模型解简单题会过度思考，提出高效优化方法。
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
- 用自训练策略减少复杂推理中的冗余计算。
- 在GSM8K等数据集上降低计算开销，保持准确率不变。
- 首次系统分析o1类模型的过拟合式推理问题。
OpenAI o1等模型凭借推理过程中模拟人类长期思考的能力表现出色，采用扩展链式思维（CoT）探索多种解题策略。然而，如何在测试时智能高效地分配计算资源仍是个关键问题。本文首次系统研究了此类模型中存在的过思考现象——对简单问题投入过多计算资源却收益甚微。我们从结果和过程两个角度引入新的效率评估指标，通过自训练范式提出缓解过思考的策略，精简推理流程而不损失准确性。实验表明，该方法在GSM8K、MATH500、GPQA和AIME等多个难度各异的测试集上均成功降低了计算开销，同时维持了模型性能。
原文摘要 · Abstract (English)
The remarkable performance of models like the OpenAI o1 can be attributed to their ability to emulate human-like long-time thinking during inference. These models employ extended chain-of-thought (CoT) processes, exploring multiple strategies to enhance problem-solving capabilities. However, a critical question remains: How to intelligently and efficiently scale computational resources during testing. This paper presents the first comprehensive study on the prevalent issue of overthinking in these models, where excessive computational resources are allocated for simple problems with minimal benefit. We introduce novel efficiency metrics from both outcome and process perspectives to evaluate the rational use of computational resources by o1-like models. Using a self-training paradigm, we propose strategies to mitigate overthinking, streamlining reasoning processes without compromising accuracy. Experimental results show that our approach successfully reduces computational overhead while preserving model performance across a range of testsets with varying difficulty levels, such as GSM8K, MATH500, GPQA, and AIME.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。