arXiv:2510.16724cs.AIcs.CL2025-10综述被引 27

用强化学习让大模型自主搜索,提升信息获取与推理能力

A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications

  • 用强化学习驱动大模型多步规划与自适应检索
  • 解决传统检索生成中静态、单轮的局限性
  • 适合研究智能代理与自主系统的研究者

大型语言模型(LLMs)通过自然语言交互革新了信息获取与推理方式,但受限于静态知识、事实幻觉以及无法获取实时或领域特定信息。检索增强生成(RAG)通过外部证据来约束模型输出,缓解上述问题,但传统RAG流程多为单轮且依赖启发式策略,缺乏对检索与推理过程的动态控制。近期的智能体搜索(agentic search)进展通过让大模型在搜索环境中进行多步交互,实现规划、检索与反思。在此范式下,强化学习(RL)提供了自适应与自我改进的搜索行为机制。本综述首次全面梳理了基于强化学习的智能体搜索,从三个维度组织该新兴领域:(i) 强化学习的功能角色,(ii) 强化学习的优化策略,(iii) 应用范围。总结代表性方法、评估协议与应用场景,并讨论开放挑战与未来方向,以构建可靠且可扩展的强化学习驱动智能体搜索系统。我们希望此综述能激发更多关于强化学习与智能体搜索融合的研究。相关资源详见 https://github.com/ventr1c/Awesome-RL-based-Agentic-Search-Papers。

原文摘要 · Abstract (English)

The advent of large language models (LLMs) has transformed information access and reasoning through open-ended natural language interaction. However, LLMs remain limited by static knowledge, factual hallucinations, and the inability to retrieve real-time or domain-specific information. Retrieval-Augmented Generation (RAG) mitigates these issues by grounding model outputs in external evidence, but traditional RAG pipelines are often single turn and heuristic, lacking adaptive control over retrieval and reasoning. Recent advances in agentic search address these limitations by enabling LLMs to plan, retrieve, and reflect through multi-step interaction with search environments. Within this paradigm, reinforcement learning (RL) offers a powerful mechanism for adaptive and self-improving search behavior. This survey provides the first comprehensive overview of \emph{RL-based agentic search}, organizing the emerging field along three complementary dimensions: (i) What RL is for (functional roles), (ii) How RL is used (optimization strategies), and (iii) Where RL is applied (scope of optimization). We summarize representative methods, evaluation protocols, and applications, and discuss open challenges and future directions toward building reliable and scalable RL driven agentic search systems. We hope this survey will inspire future research on the integration of RL and agentic search. Our repository is available at https://github.com/ventr1c/Awesome-RL-based-Agentic-Search-Papers.

强化学习智能体搜索大模型RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。