用自我反思提升搜索智能体深度，让模型更准更会思考。
WebSeer: Training Deeper Search Agents through Reinforcement Learning with Self-Reflection
- 引入自我反思机制，训练更深层次的交互式搜索
- 在HotpotQA和SimpleQA上分别达72.3%和90.0%准确率
- 适合需要长链推理与高可靠性的信息检索场景
搜索智能体在交互环境中显著提升了信息检索与决策能力。尽管强化学习已用于训练具备动态交互检索能力的智能体，但现有方法受限于浅层工具使用深度及多轮交互中错误累积。本文提出WebSeer，一种通过自反思机制增强的强化学习搜索智能体。我们构建了包含反思模式的大规模标注数据集，并设计两阶段训练框架，在真实网络环境中共融冷启动与强化学习，使模型生成更长、更深刻的工具使用轨迹。该方法显著延长工具使用链并提升答案准确性。仅用一个14B参数模型，在HotpotQA和SimpleQA上分别达到72.3%和90.0%的准确率，并展现出对分布外数据的强大泛化能力。代码已开源。
原文摘要 · Abstract (English)
Search agents have achieved significant advancements in enabling intelligent information retrieval and decision-making within interactive environments. Although reinforcement learning has been employed to train agentic models capable of more dynamic interactive retrieval, existing methods are limited by shallow tool-use depth and the accumulation of errors over multiple iterative interactions. In this paper, we present WebSeer, a more intelligent search agent trained via reinforcement learning enhanced with a self-reflection mechanism. Specifically, we construct a large dataset annotated with reflection patterns and design a two-stage training framework that unifies cold start and reinforcement learning within the self-reflection paradigm for real-world web-based environments, which enables the model to generate longer and more reflective tool-use trajectories. Our approach substantially extends tool-use chains and improves answer accuracy. Using a single 14B model, we achieve state-of-the-art results on HotpotQA and SimpleQA, with accuracies of 72.3% and 90.0%, respectively, and demonstrate strong generalization to out-of-distribution datasets. The code is available at https://github.com/99hgz/WebSeer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。