用图结构分析网页代理行为，发现传统评估忽略的效率问题。
WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation
- 将多轮操作抽象为加权动作图，捕捉行为结构差异。
- 在六种代理上分析数千条轨迹，揭示冗余与关键决策点。
- 适合关注代理效率与行为多样性的研究者使用。
当前网页代理评估多依赖二元成功指标或单一参考轨迹，忽略了基准数据集中存在的结构多样性。我们提出WebGraphEval框架，将多个代理的轨迹抽象为统一的加权动作图，兼容WebArena等基准，无需修改环境即可利用排行榜数据和新收集轨迹。该框架规范编码动作，合并重复行为，并应用奖励传播与成功率加权边统计等结构分析方法。对六种网页代理的数千条轨迹评估显示，图抽象能捕捉跨模型规律，揭示冗余与低效行为，并识别出结果指标遗漏的关键决策点。通过将网页交互建模为图结构数据，WebGraphEval建立了一种面向多路径、跨代理及效率感知的通用评估方法。
原文摘要 · Abstract (English)
Current evaluation of web agents largely reduces to binary success metrics or conformity to a single reference trajectory, ignoring the structural diversity present in benchmark datasets. We present WebGraphEval, a framework that abstracts trajectories from multiple agents into a unified, weighted action graph. This representation is directly compatible with benchmarks such as WebArena, leveraging leaderboard runs and newly collected trajectories without modifying environments. The framework canonically encodes actions, merges recurring behaviors, and applies structural analyses including reward propagation and success-weighted edge statistics. Evaluations across thousands of trajectories from six web agents show that the graph abstraction captures cross-model regularities, highlights redundancy and inefficiency, and identifies critical decision points overlooked by outcome-based metrics. By framing web interaction as graph-structured data, WebGraphEval establishes a general methodology for multi-path, cross-agent, and efficiency-aware evaluation of web agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。