arXiv:2509.06501cs.CL2025-09被引 78

用迭代查询演化生成复杂数据,训练出能长时思考的网页智能体。

WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents

  • 通过模型探索和查询逐步简化,自动生成多步推理的难题数据。
  • 80亿参数模型平均完成16次操作,性能超720亿参数模型。
  • 适合研究长序列任务、网页代理和强化学习的开发者参考。

大型语言模型正转向智能体应用,网页浏览能力是获取多样网络信息的关键。然而现有开源网页智能体或信息搜索能力有限,或实现不透明。本文指出核心挑战在于信息搜索任务数据稀缺。为此,提出WebExplorer:一种基于模型探索与迭代查询演化的系统性数据生成方法,创建需多步推理和复杂导航的挑战性问答对。利用构建的高质量数据集,通过监督微调和强化学习训练出先进的网页智能体WebExplorer-8B,支持128K上下文长度及最多100次工具调用,实现长周期问题求解。在多个信息搜索基准测试中,该模型在同等规模下达到当前最优表现。尤其作为80亿参数模型,经强化学习后平均可完成16次操作,在BrowseComp-en/zh上准确率超过720亿参数的WebSailor-72B;在WebWalkerQA和FRAMES上,其表现优于所有不超过1000亿参数的模型。此外,尽管仅在知识密集型问答数据上训练,其在HLE基准测试中也展现出强泛化能力。这些结果验证了本方法对构建长周期网页智能体的实用性。

原文摘要 · Abstract (English)

The paradigm of Large Language Models (LLMs) has increasingly shifted toward agentic applications, where web browsing capabilities are fundamental for retrieving information from diverse online sources. However, existing open-source web agents either demonstrate limited information-seeking abilities on complex tasks or lack transparent implementations. In this work, we identify that the key challenge lies in the scarcity of challenging data for information seeking. To address this limitation, we introduce WebExplorer: a systematic data generation approach using model-based exploration and iterative, long-to-short query evolution. This method creates challenging query-answer pairs that require multi-step reasoning and complex web navigation. By leveraging our curated high-quality dataset, we successfully develop advanced web agent WebExplorer-8B through supervised fine-tuning followed by reinforcement learning. Our model supports 128K context length and up to 100 tool calling turns, enabling long-horizon problem solving. Across diverse information-seeking benchmarks, WebExplorer-8B achieves the state-of-the-art performance at its scale. Notably, as an 8B-sized model, WebExplorer-8B is able to effectively search over an average of 16 turns after RL training, achieving higher accuracy than WebSailor-72B on BrowseComp-en/zh and attaining the best performance among models up to 100B parameters on WebWalkerQA and FRAMES. Beyond these information-seeking tasks, our model also achieves strong generalization on the HLE benchmark even though it is only trained on knowledge-intensive QA data. These results highlight our approach as a practical path toward long-horizon web agents.

网页智能体长时序推理数据生成强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。