提出新型推理学习框架，解决大模型从思维链数据中高效学习的难题。
From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
- 将推理建模为带验证器的深度优先搜索，支持失败回溯。
- 在两种合理假设下，证明其可高效学习而现有方法失效。
- 适合构建可扩展、可解释的复杂推理系统，如大型推理模型。
思维链（CoT）推理已成为提升大语言模型（LLM）问题求解能力的强大工具。然而，从CoT数据中学习的理论基础仍不完善，现有方法如监督微调（SFT）、强化学习（RL）、思维树（ToT）和蒙特卡洛树搜索（MCTS）在复杂推理任务上常表现不佳。本文识别出阻碍有效CoT学习的核心障碍：分布漂移、缺乏嵌入式搜索机制以及指数级推理成本。为此，我们提出「勤勉学习者」（Diligent Learner）这一新学习范式，显式将推理建模为由验证器引导的深度优先搜索，并支持失败时回溯。在两个温和且现实的假设下，我们证明该框架能高效从CoT数据中学习，而现有方法无法做到。该框架为基于自然产生、不完整数据训练可扩展、可靠的推理系统提供了路径，助力实现具备强大、可解释问题求解能力的大型推理模型（LRMs）。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning has emerged as a powerful tool for enhancing the problem-solving capabilities of large language models (LLMs). However, the theoretical foundations of learning from CoT data remain underdeveloped, and existing approaches -- such as Supervised Fine-Tuning (SFT), Reinforcement Learning (RL), Tree-of-Thoughts (ToT), and Monte Carlo Tree Search (MCTS) -- often fail on complex reasoning tasks. In this work, we identify core obstacles that hinder effective CoT learning, including distribution drift, lack of embedded search, and exponential inference costs. We introduce the Diligent Learner, a new learning paradigm that explicitly models reasoning as a depth-first search guided by a validator and supports backtracking upon failure. Under two mild and realistic assumptions, we prove that the Diligent Learner can efficiently learn from CoT data while existing methods fail to do so. This framework offers a path toward building scalable and reliable reasoning systems trained on naturally occurring, incomplete data -- paving the way for the development of Large Reasoning Models (LRMs) with robust, interpretable problem-solving abilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。