系统梳理GUI智能体的效率问题与优化方法
Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
- 从观察、记忆、动作到运行时，全链路分析效率瓶颈
- 提出选择性读取、可恢复记忆等五类核心优化机制
- 适合关注自动化工具部署效率的研究者和开发者
GUI智能体在网页、移动应用和桌面环境中日益普及，但当前研究仍主要依赖任务成功率评估。我们指出，实际部署的效率同样关键：智能体在成功完成任务的同时，应尽可能减少上下文占用、计算开销、动作预算及运行时延迟。本综述从端到端系统视角出发，涵盖观察效率、上下文与记忆效率、动作效率以及规划侧/系统级效率四大维度。通过定向搜索及引文追溯扩展原始文献，提炼出主流机制、报告的效率指标及其引入的新开销。研究表明，近期进展集中于五大核心思想：选择性读取而非全量上下文摄入、全局到局部视觉分配、可恢复记忆而非原始历史重放、验证感知控制，以及支持在GUI与非GUI模式间切换的混合运行时。最后，本文识别出若干开放问题：验证器成本的透明核算、跨基准可比性，以及在真实延迟与隐私约束下对观察、记忆与执行层的协同设计。
原文摘要 · Abstract (English)
GUI agents increasingly operate across websites, mobile apps, and desktop environments, yet the field still reports progress primarily through task success. We argue that practical deployment depends equally on efficiency: how much context, computation, action budget, and runtime overhead an agent consumes while succeeding. This survey studies efficient GUI agents through an end-to-end systems lens that preserves the current technical axes of observation efficiency, context and memory efficiency, action efficiency, and planner-side/system efficiency. For each subsection, we expand the seed literature through targeted search plus backward and forward citation chaining, then synthesize the dominant mechanisms, reported efficiency signals, and new overheads they introduce. Across the literature, recent progress converges on a small set of recurring ideas: selective reading instead of full-context ingestion, global-to-local visual allocation, recoverable memory rather than raw history replay, verification-aware control, and hybrid runtimes that can switch between GUI and non-GUI execution. We conclude by identifying the main open problems, including honest accounting of verifier cost, cross-benchmark comparability, and co-design of observation, memory, and execution layers under real latency and privacy constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。