为大模型网络代理提供可诊断的规划框架，揭示不同架构优劣。
AI Planning Framework for LLM-Based Web Agents
- 将代理架构映射到经典搜索算法，实现失败原因系统化分析
- 在794条人类标注轨迹上验证，全计划先行方案元素准确率达89%
- 提出新评估指标，帮助根据任务需求选型合适代理架构
开发自主执行网络任务的智能体是人工智能的核心挑战。尽管大语言模型代理能理解复杂用户请求，但其常表现为黑箱，难以诊断失败原因或规划过程。本文将网络任务形式化为序列决策问题，提出一个分类体系：逐步代理对应广度优先搜索（BFS），树搜索代理对应最佳优先树搜索，预先规划代理对应深度优先搜索（DFS）。该框架支持对上下文漂移、任务分解不一致等问题的系统诊断。为此，我们设计了五项新颖评估指标，超越简单成功率，衡量轨迹质量。基于WebArena基准的794条人工标注轨迹数据集进行验证。结果表明，虽然逐步代理更贴近人类黄金轨迹（38%成功率），但预先规划代理在元素准确率（89%）等技术指标上表现更优，证明了新评估体系对适配特定应用需求的重要性。
原文摘要 · Abstract (English)
Developing autonomous agents for web-based tasks is a core challenge in AI. While Large Language Model (LLM) agents can interpret complex user requests, they often operate as black boxes, making it difficult to diagnose why they fail or how they plan. This paper addresses this gap by formally treating web tasks as sequential decision-making processes. We introduce a taxonomy that maps modern agent architectures to traditional planning paradigms: Step-by-Step agents to Breadth-First Search (BFS), Tree Search agents to Best-First Tree Search, and Full-Plan-in-Advance agents to Depth-First Search (DFS). This framework allows for a principled diagnosis of system failures like context drift and incoherent task decomposition. To evaluate these behaviors, we propose five novel evaluation metrics that assess trajectory quality beyond simple success rates. We support this analysis with a new dataset of 794 human-labeled trajectories from the WebArena benchmark. Finally, we validate our evaluation framework by comparing a baseline Step-by-Step agent against a novel Full-Plan-in-Advance implementation. Our results reveal that while the Step-by-Step agent aligns more closely with human gold trajectories (38% overall success), the Full-Plan-in-Advance agent excels in technical measures such as element accuracy (89%), demonstrating the necessity of our proposed metrics for selecting appropriate agent architectures based on specific application constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。