让大模型学会自我回溯，提升推理效率和准确性
Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models
- 提出自回溯机制，让模型自主决定何时回退
- 推理性能提升超40%，优于最优路径监督微调
- 适合需要强逻辑推理的AI系统研发者
将慢思考机制融入大语言模型（LLMs）是实现二级通用智能推理能力的有前景路径，如OpenAI的o1所示。然而，仍存在过度思考效率低、过度依赖辅助奖励模型等挑战。我们指出，这些限制源于LLM无法内化搜索过程这一关键环节。解决此问题的关键是赋予模型在训练与推理中自主判断回溯时机的能力，这是传统搜索算法的核心操作。为此，我们提出自回溯机制，使LLM具备在推理过程中自我回溯的能力。该机制不仅增强推理能力，还通过自我优化将慢思考转化为快思考，提升效率。实证评估表明，该方法显著提升LLM推理能力，在多个任务上较最优路径监督微调方法性能提升超过40%。本研究为构建更先进、更稳健的推理系统提供了新路径。
原文摘要 · Abstract (English)
The integration of slow-thinking mechanisms into large language models (LLMs) offers a promising way toward achieving Level 2 AGI Reasoners, as exemplified by systems like OpenAI's o1. However, several significant challenges remain, including inefficient overthinking and an overreliance on auxiliary reward models. We point out that these limitations stem from LLMs' inability to internalize the search process, a key component of effective reasoning. A critical step toward addressing this issue is enabling LLMs to autonomously determine when and where to backtrack, a fundamental operation in traditional search algorithms. To this end, we propose a self-backtracking mechanism that equips LLMs with the ability to backtrack during both training and inference. This mechanism not only enhances reasoning ability but also efficiency by transforming slow-thinking processes into fast-thinking through self-improvement. Empirical evaluations demonstrate that our proposal significantly enhances the reasoning capabilities of LLMs, achieving a performance gain of over 40 percent compared to the optimal-path supervised fine-tuning method. We believe this study introduces a novel and promising pathway for developing more advanced and robust Reasoners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。