自动生成海量真实网页环境，提升GUI智能体训练效果
InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
- 用统一规范+任务驱动开发生成连贯网页
- 生成环境使智能体在两个基准上表现更优
- 适合做界面交互智能体的研究者与开发者
代表用户与图形界面交互的GUI智能体是实用型AI助手的重要方向，但训练受限于合适环境的稀缺。本文提出InfiniteWeb系统，可规模化自动生成功能性网页环境用于GUI智能体训练。尽管大模型在生成单个网页方面表现良好，但构建多个相互关联的完整网站仍面临挑战。为此，我们采用统一规范、任务导向的测试驱动开发，并结合网站种子与参考设计图以保证多样性。系统还生成可验证的任务评估器，为强化学习提供密集奖励信号。实验表明，InfiniteWeb在真实网站构建上优于商业编码智能体，基于其生成环境训练的GUI智能体在OSWorld和Online-Mind2Web基准上均取得显著性能提升，验证了该系统的有效性。
原文摘要 · Abstract (English)
GUI agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants. However, training such agents is hindered by the scarcity of suitable environments. We present InfiniteWeb, a system that automatically generates functional web environments at scale for GUI agent training. While LLMs perform well on generating a single webpage, building a realistic and functional website with many interconnected pages faces challenges. We address these challenges through unified specification, task-centric test-driven development, and a combination of website seed with reference design image to ensure diversity. Our system also generates verifiable task evaluators enabling dense reward signals for reinforcement learning. Experiments show that InfiniteWeb surpasses commercial coding agents at realistic website construction, and GUI agents trained on our generated environments achieve significant performance improvements on OSWorld and Online-Mind2Web, demonstrating the effectiveness of proposed system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。