系统梳理大模型推理前沿,涵盖推理增强、学习推理与智能体系统。
A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
- 按推理时机与系统架构分类,构建双维分析框架
- 揭示从推理扩展到学习推理的演进趋势,如DeepSeek-R1
- 适合关注AI智能体、推理机制研究者阅读
推理是实现逻辑推断、问题求解与决策的核心认知过程。随着大语言模型(LLMs)的快速发展,推理能力成为先进AI系统区别于传统聊天机器人的重要标志。本文从两个正交维度对现有方法进行分类:(1) 推理阶段,即推理是在推理阶段实现还是通过专门训练获得;(2) 系统架构,区分独立大模型与集成外部工具或多智能体协作的复合系统。每个维度下进一步考察输入层面(如构建高质量提示)与输出层面(如优化多个采样结果)的关键技术。该分类体系系统呈现了大模型推理的发展脉络,凸显了从推理扩展向学习推理(如DeepSeek-R1)转变、以及向智能体工作流(如OpenAI Deep Research、Manus Agent)演进的趋势。此外,还涵盖从监督微调到强化学习(如PPO、GRPO)等广泛训练算法,以及推理器与验证器的训练方法。同时分析智能体工作流的关键设计,包括生成-评估、大模型辩论等经典模式与最新创新。
原文摘要 · Abstract (English)
Reasoning is a fundamental cognitive process that enables logical inference, problem-solving, and decision-making. With the rapid advancement of large language models (LLMs), reasoning has emerged as a key capability that distinguishes advanced AI systems from conventional models that empower chatbots. In this survey, we categorize existing methods along two orthogonal dimensions: (1) Regimes, which define the stage at which reasoning is achieved (either at inference time or through dedicated training); and (2) Architectures, which determine the components involved in the reasoning process, distinguishing between standalone LLMs and agentic compound systems that incorporate external tools, and multi-agent collaborations. Within each dimension, we analyze two key perspectives: (1) Input level, which focuses on techniques that construct high-quality prompts that the LLM condition on; and (2) Output level, which methods that refine multiple sampled candidates to enhance reasoning quality. This categorization provides a systematic understanding of the evolving landscape of LLM reasoning, highlighting emerging trends such as the shift from inference-scaling to learning-to-reason (e.g., DeepSeek-R1), and the transition to agentic workflows (e.g., OpenAI Deep Research, Manus Agent). Additionally, we cover a broad spectrum of learning algorithms, from supervised fine-tuning to reinforcement learning such as PPO and GRPO, and the training of reasoners and verifiers. We also examine key designs of agentic workflows, from established patterns like generator-evaluator and LLM debate to recent innovations. ...
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。