arXiv:2504.21024cs.CL2025-04EMNLP被引 48

通过共进化世界模型,让网页智能体持续自我优化,突破性能瓶颈。

WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model

  • 引入共进化世界模型预测网页下一步状态,生成自指导训练数据。
  • 在真实网页环境上提升10%性能,无需依赖闭源模型蒸馏。
  • 适合研究自主智能体、自进化系统和大模型应用的开发者。

智能体自我改进是一种新兴方法,通过自主采样轨迹并基于自身策略训练底层大语言模型(LLM),以提升性能。近期在网页环境中的进展面临关键瓶颈:自主学习周期中性能趋于停滞,难以进一步提升。我们认为这源于对网页环境探索不足,以及未能充分挖掘LLM中预训练的网页知识。为此,我们提出一种新框架,引入共进化世界模型LLM,该模型基于当前观察和动作预测下一状态。利用LLM对海量网页内容的预训练知识,世界模型发挥双重作用:(1) 作为虚拟网页服务器,生成自指导训练数据,持续优化智能体策略;(2) 在推理阶段作为想象引擎,进行前瞻模拟,指导智能体LLM的动作选择。在真实网页环境(Mind2Web-Live、WebVoyager 和 GAIA-web)上的实验表明,相比现有自演化智能体,性能提升10%,验证了方法的有效性与泛化能力,且未使用任何来自更强大闭源模型的蒸馏。本工作确立了将世界模型整合进自主智能体框架的必要性,以实现持续适应性。代码已开源:https://github.com/Tencent/SelfEvolvingAgent

原文摘要 · Abstract (English)

Agent self-improvement, where the backbone Large Language Model (LLM) of the agent are trained on trajectories sampled autonomously based on their own policies, has emerged as a promising approach for enhancing performance. Recent advancements, particularly in web environments, face a critical limitation: their performance will reach a stagnation point during autonomous learning cycles, hindering further improvement. We argue that this stems from limited exploration of the web environment and insufficient exploitation of pre-trained web knowledge in LLMs. To improve the performance of self-improvement, we propose a novel framework that introduces a co-evolving World Model LLM. This world model predicts the next observation based on the current observation and action within the web environment. Leveraging LLMs' pretrained knowledge of abundant web content, the World Model serves dual roles: (1) as a virtual web server generating self-instructed training data to continuously refine the agent's policy, and (2) as an imagination engine during inference, enabling look-ahead simulation to guide action selection for the agent LLM. Experiments in real-world web environments (Mind2Web-Live, WebVoyager, and GAIA-web) show a 10% performance gain over existing self-evolving agents, demonstrating the efficacy and generalizability of our approach, without using any distillation from more powerful close-sourced models. Our work establishes the necessity of integrating world models into autonomous agent frameworks to unlock sustained adaptability. Code is available at https://github.com/Tencent/SelfEvolvingAgent

智能体自进化世界模型网页任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。