测试网页代理在网页变化下的表现，发现其易受界面更新影响。
TimeWarp: Evaluating Web Agents by Revisiting the Past
- 用容器化环境模拟不同时期网页界面，构建动态测试基准。
- 单一版本训练的代理在新界面下性能下降，改进后提升至37.7%。
- 提出跨版本计划蒸馏方法，适合研究网页代理鲁棒性者使用。
当前网页代理在基准测试中表现提升，引发疑问：当网页发生变化时，这些代理是否仍能保持同样表现？我们提出TimeWarp，一个通过容器化环境模拟不断演化的网页的基准测试，包含三个网页环境，每个环境有六个跨越互联网不同时期的界面版本，并配以复杂、真实的导航任务。实验表明，网页代理对界面变化极为敏感，行为克隆（BC）在单一版本轨迹上效果有限。为此，我们提出TimeTraj，一种简单有效的算法，通过计划蒸馏在多个版本中收集轨迹。利用该方法，基于教师回放的BC变体训练，使Qwen-3 4B模型性能从20.4%提升至37.7%，Llama-3.1 8B模型从0%提升至27.0%。我们希望本工作能推动研究者探索跨网页设计的泛化能力，并开启以收集计划而非轨迹为新范式，提升网页代理的鲁棒性。
原文摘要 · Abstract (English)
The improvement of web agents on current benchmarks raises the question: Do today's agents perform just as well when the web changes? We introduce TimeWarp, a benchmark that emulates the evolving web using containerized environments that vary in UI, design, and layout. TimeWarp consists of three web environments, each with six UI versions spanning different eras of the internet, paired with a set of complex, realistic tasks requiring different forms of web navigation. Our experiments reveal web agents' vulnerability to changes and the limitations of behavior cloning (BC) on single-version trajectories. To address this, we propose TimeTraj, a simple yet effective algorithm that uses plan distillation to collect trajectories across multiple versions. By training agents on teacher rollouts using our BC-variant, we achieve substantial performance gains: $20.4\%\rightarrow37.7\%$ for Qwen-3 4B and $0\%\rightarrow27.0\%$ for Llama-3.1 8B models. We hope our work helps researchers study generalization across web designs and unlock a new paradigm for collecting plans rather than trajectories, thereby improving the robustness of web agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。