发现网页智能体完成任务却仍出错，揭示三类隐藏失败模式。
When Web Agents Finish but Still Fail: Reproducible Triggers and Trace Diagnostics for Parallel Web Exploration
- 用并行网页探索基准测试智能体行为，识别隐性错误。
- 合成数据微调后完成率升至96.0%，但正确率仅0.4529。
- 发现搜索循环、过早终止、证据合并失败三大致命缺陷。
长周期网页智能体常在最终答案评估中隐藏失败：它们可能访问了有用页面，生成了格式正确的回答，并自信结束，但仍遗漏关键字段、包含无支持项或依赖过时证据。我们通过并行网页探索基准 Parallel WebBench（含1,679条验证记录：350个手工构建的并行任务和1,329条基于URL轨迹重建的记录）研究此类问题。在16k上下文和16轮交互下，使用人类数据、均衡人机合成数据及合成数据为主的混合训练策略，对WebExplorer风格智能体进行GRPO微调。最佳GRPO模型将完成率从WebExplorer-8B的50.7%提升至96.0%，元素级F1值从GPT-4.1-mini评判的0.2489提升至0.4529，但二元准确率仍远低于完成率。追踪分析揭示三种持续存在的失败模式：上下文绑定的搜索循环、在部分答案上过早终止、在已获取相关证据后发生合成崩溃。结果表明，合成数据驱动的GRPO虽减少回避行为并提升部分正确性，但完成率与正确率之间仍存在显著差距，需依赖证据覆盖与合成诊断来解决。
原文摘要 · Abstract (English)
Long-horizon web agents often fail in ways hidden by final-answer evaluation: they may visit useful pages, produce a well-formed answer, and terminate confidently while still missing fields, over-including unsupported items, or relying on stale evidence. We study these failures with Parallel WebBench, a parallel web-exploration benchmark containing 1,679 verified records: 350 manually curated parallel tasks and 1,329 reconstructed records with verified URL-based trajectories. We train WebExplorer-style agents with GRPO under human-only, balanced human-synthetic, and synthetic-heavy data mixtures. At 16k context and 16 interaction rounds, the best GRPO model improves completion over WebExplorer-8B from 50.7% to 96.0% and GPT-4.1-mini-judged element-wise F1 from 0.2489 to 0.4529, but binary accuracy remains far below completion. Trace-level analysis identifies three persistent failure modes: context-bound search loops, premature termination on partial answers, and synthesis collapse after relevant evidence has already been retrieved. These results show that synthetic-data GRPO reduces abstention and improves partial correctness, but leaves a completion-correctness gap that requires evidence-grounded coverage and synthesis diagnostics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。