AutoSurfer通过系统探索生成高质量网页轨迹,提升代理任务准确性。
AutoSurfer -- Teaching Web Agents through Comprehensive Surfing, Learning, and Modeling

- 采用广度优先探索策略,模拟人类学习网站过程,避免重复访问。
- 基于真实浏览路径合成任务,减少幻觉,提升轨迹可靠性。
- 适合训练针对特定网站的LLM代理,尤其在复杂任务上表现更优。
多模态大语言模型的进展推动了网页代理自动化复杂任务的能力,但其准确性受限于高质量网页轨迹数据的稀缺。现有自动轨迹生成方法因依赖首页任务提议或随机探索,导致网站覆盖不全,常产生幻觉或模糊的任务合成,进而造成轨迹不完整且不可靠。本文提出AutoSurfer,通过三项关键创新解决上述问题:第一,采用系统性广度优先探索策略,维护已发现页面与操作轨迹队列,跨页面传播知识以避免冗余探索,并递归展开多层图形界面元素,贴近人类学习新网站的方式;第二,利用探索轨迹引导任务合成,将复杂任务基于实际导航路径而非孤立动作或页面内容,降低幻觉;第三,使用相同探索轨迹作为提示,指导网页代理进行更准确、可靠的轨迹优化。这些创新使AutoSurfer全面覆盖网站动作空间,生成适用于训练特定网站LLM的数据。我们在WebArena基准上微调Qwen2.5-VL-7B-Instruct进行评估,结果表明,相比最先进方法Explorer、OS-Genesis和SynthAgent,AutoSurfer在任务完成率上最高提升至24.23%,优于最佳前序方法的19.59%。进一步的任务多样性分析显示,AutoSurfer生成的任务分布更具多样性。
原文摘要 · Abstract (English)
Recent advances in multimodal large language models (LLMs) have revolutionized web agents that can automate complex tasks on websites. However, their accuracy remains limited by the scarcity of high-quality web trajectory training data. Existing automatic trajectory generation methods suffer from incomplete website coverage due to homepage-based task proposals or random-walk exploration. Such methods often result in hallucinated or ambiguous task synthesis that lead to incomplete and unreliable trajectory generation. Here, we present AutoSurfer, a comprehensive web trajectory generator that addresses these limitations through three key innovations. First, AutoSurfer employs a systematic breadth-first exploration strategy that maintains a queue of discovered pages and action traces, propagates knowledge across pages to avoid redundant exploration, and recursively expands multi-level graphical user interface elements - closely resembling how a human would learn a new website. Second, AutoSurfer leverages the exploration trajectory to guide task synthesis, reducing hallucinations by grounding complex tasks in actual navigation paths rather than isolated actions or page content alone. Third, AutoSurfer uses the same exploration trajectory as hints to steer a web agent toward more accurate and reliable trajectory refinement. Together, these innovations enable AutoSurfer to comprehensively cover a website's action space and generate data suitable for training website-specific LLMs. We evaluate AutoSurfer on the WebArena benchmark by fine-tuning Qwen2.5-VL-7B-Instruct and demonstrate that it outperforms state-of-the-art methods - Explorer, OS-Genesis, and SynthAgent - achieving up to 24.23% overall task completion accuracy compared to 19.59% for the best prior method. Further, task diversity analysis demonstrates that AutoSurfer yields a more diverse distribution of synthesized tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。