用世界模型模拟网页环境,高效生成高质量操作轨迹。
WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis
- 基于世界模型构建虚拟网页环境,支持快速树搜索规划。
- 在小规模合成数据上训练的代理性能媲美甚至超越真实数据训练者。
- 适合需要低成本、可复现训练的自动化网页交互研究者。
大型语言模型(LLM)的进展显著提升了网页代理的能力。然而,在复杂动态网页环境中进行有效导航仍需更先进的轨迹级规划与执行。以往研究通过真实环境交互收集大量GUI轨迹实现自提升,但面临两大挑战:(1) 环境状态不可控,真实或沙盒网页环境常产生不稳定、非确定性反馈,难以复现和调试代理行为;(2) 高API成本,单条交互轨迹可能涉及数百次调用,导致显著的API使用与计算开销。为解决这些局限并实现代理的可扩展自提升,我们提出WebSynthesis,一种用于轨迹合成与训练的新框架。WebSynthesis利用学习到的世界模型模拟虚拟网页环境,使策略代理能进行高效且可逆的树形规划。该方法支持大规模生成多样化、高质量的轨迹,随后用于优化代理策略。实验表明,仅在小规模合成数据上训练的代理,其性能可媲美甚至超越在大规模真实数据上训练的模型。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have significantly improved the capabilities of web agents. However, effectively navigating complex and dynamic web environments still requires more advanced trajectory-level planning and execution. Prior studies have addressed self-improving agents by collecting extensive GUI trajectories from real-environment interactions. Despite their effectiveness, these approaches encounter two critical challenges: (1) Uncontrollable environment states, where real or sandboxed web environments often yield unstable and non-deterministic feedback, complicating the reproduction and debugging of agent behaviors; and (2) High API costs, as generating even a single interaction trajectory can involve hundreds of queries, leading to considerable API usage and computational expenses. To address these limitations and enable scalable self-improvement for agents, we propose WebSynthesis, a novel framework for trajectory synthesis and training. WebSynthesis leverages a learned world model to simulate virtual web environments, allowing a policy agent to perform efficient and reversible tree-based planning. This approach supports the large-scale generation of diverse and high-quality trajectories, which are subsequently utilized to refine the agent's policy. Experimental results demonstrate that an agent trained using WebSynthesis on a small-scale synthetic dataset achieves performance comparable to or even surpassing that of models trained on large-scale real-world data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。