让图形界面智能体能回溯错误路径，提升复杂任务成功率。
BEAP-Agent: Backtrackable Execution and Adaptive Planning for GUI Agents
- 将界面操作建模为深度优先搜索，支持多层级状态回退。
- 在OSWorld基准上实现28.2%的准确率，优于现有方法。
- 适合需要长期任务探索与容错的自动化场景。
图形界面智能体旨在自动化重复性任务并提升效率。然而，现有方法在执行错误路径后难以恢复，常导致任务失败。本文将GUI任务执行建模为深度优先搜索过程，提出基于DFS的BEAP-Agent框架，支持长距离、多层次的状态回溯，并具备动态任务跟踪与更新能力。该框架由规划器、执行器和追踪器三部分协同构成，有效支持任务探索与执行。BEAP-Agent填补了GUI智能体系统化回溯机制的空白，为长周期任务探索提供解决方案。我们在OSWorld基准上进行了系统评估，结果显示其准确率达28.2%，验证了方法的有效性。
原文摘要 · Abstract (English)
GUI agents are designed to automate repetitive tasks and enhance productivity. However, existing GUI agents struggle to recover once they follow an incorrect exploration path, often leading to task failure. In this work, we model GUI task execution as a DFS process and propose BEAP-Agent, a DFS-based framework that supports long-range, multi-level state backtracking with dynamic task tracking and updating. The framework consists of three collaborative components: Planner, Executor, and Tracker. Together, they enable effective task exploration and execution. BEAP-Agent fills the gap in systematic backtracking mechanisms for GUI agents, offering a systematic solution for long-horizon task exploration. We conducted a systematic evaluation on the OSWorld benchmark, where BEAP-Agent achieved an accuracy of 28.2%, validating the effectiveness of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。