arXiv:2606.12485cs.LGcs.AI2026-06

提出分支回滚修正机制,解决网页代理模仿学习中的干预时机难题。

Speculative Rollback Correction for Quality-Diverse Web Agent Imitation

论文配图:Speculative Rollback Correction for Quality-Diverse Web Agent Imitation
图 1 · 摘自论文原文
  • 采用固定时域分支回顾,先试运行再由教师定位首个有害偏差
  • 收集977条验证通过轨迹和9183个下一步动作样本,提升数据质量与多样性
  • 适合需要高鲁棒性和多样化策略的网页自动化任务

通过专家轨迹进行模仿学习来训练交互式网页代理已成为一种高效方法。然而,如何确定最优的专家干预时机是关键挑战:延迟干预导致早期错误累积,使页面状态进入不可恢复状态;过早或过度干预则使代理过度依赖专家策略,陷入单一僵化路径的局部最优。我们提出推测性回滚修正(SRC),一种适用于可重置环境的分支级模仿框架。不同于在每个状态请求教师标签或仅在完整轨迹后修正,SRC采用固定时域分支审查:学生先执行一段短期推测性片段,教师仅在局部进展中断时定位首个有害偏差。回滚保留有效前缀,成功回放由硬验证器筛选并存入轻量级质量-多样性档案。所获数据支持对局部修正与验证通过轨迹的下一步动作监督微调。在WebArena-Infinity上,SRC收集了977条验证通过轨迹和9183个下一步动作示例;固定时域审查相比逐步审查,在恢复能力与查询频率间取得更好权衡,同时保留验证通过的解法变体。代码已公开于https://github.com/LongkunHao/SRC_gui_agent。

原文摘要 · Abstract (English)

Training interactive web agents through imitation learning from expert trajectories has emerged as a highly effective approach. However, determining the optimal timing for expert intervention presents a critical challenge in this context. Delayed intervention often leads to the accumulation of early-stage errors, pushing the page state into an irrecoverable regime. Conversely, premature or excessive intervention causes the agent to become overly reliant on expert policies, trapping the model in local optima characterized by a single, rigid trajectory. We propose Speculative Rollback Correction (SRC), a branch-level imitation framework for resettable agent environments. Instead of requesting teacher labels at every visited state or correcting only after a completed trajectory, SRC uses fixed-horizon branch review: the student executes a short speculative segment before teacher review, and the teacher localizes the first harmful deviation only when local progress breaks. Rollback preserves useful prefixes, while successful rollouts are filtered by a hard verifier and retained in a lightweight quality-diversity archive. The resulting data supports next-action supervised fine-tuning on both localized corrections and verifier-passing trajectories. On WebArena-Infinity, SRC collects 977 verifier-passing trajectories and 9,183 next-action examples; fixed-horizon review improves the recovery-versus-query tradeoff over step-level review while retaining verifier-passing solution variants. Code is available at https://github.com/LongkunHao/SRC_gui_agent.

模仿学习网页代理质量多样性纠错机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。