通过意图抽象与共享记忆,提升电脑操作代理的长期执行稳定性。
IntentCUA: Learning Intent-level Representations for Skill Abstraction and Multi-Agent Planning in Computer-Use Agents
- 用多视角意图表征和共享记忆抽象操作轨迹,生成可复用技能。
- 端到端测试中任务成功率74.83%,步效率比达0.91,优于基线方法。
- 适合长周期、复杂桌面场景下的自动化系统设计与优化。
计算机使用代理在长时程、感知噪声大、多窗口上下文和动态环境状态下运行。现有方法(如基于强化学习的规划或轨迹检索)常偏离用户意图,重复解决常规子问题,导致误差累积和效率低下。本文提出IntentCUA,一种多智能体电脑操作框架,通过意图对齐的计划记忆稳定长时程执行。规划器、计划优化器与批评者协同工作,共享内存将原始交互轨迹抽象为多视角意图表示和可复用技能。运行时,意图原型检索子组对齐的技能并注入部分计划,减少冗余重规划,缓解跨应用的误差传播。端到端评估显示,IntentCUA任务成功率达74.83%,步效率比为0.91,优于基于强化学习和轨迹中心的基线。消融实验表明,多视角意图抽象与共享计划记忆协同提升执行稳定性,合作式多智能体循环在长时程任务中带来最大增益。结果表明,系统级意图抽象与记忆驱动的协调是实现大型动态环境中可靠高效桌面自动化的关键。
原文摘要 · Abstract (English)
Computer-use agents operate over long horizons under noisy perception, multi-window contexts, evolving environment states. Existing approaches, from RL-based planners to trajectory retrieval, often drift from user intent and repeatedly solve routine subproblems, leading to error accumulation and inefficiency. We present IntentCUA, a multi-agent computer-use framework designed to stabilize long-horizon execution through intent-aligned plan memory. A Planner, Plan-Optimizer, and Critic coordinate over shared memory that abstracts raw interaction traces into multi-view intent representations and reusable skills. At runtime, intent prototypes retrieve subgroup-aligned skills and inject them into partial plans, reducing redundant re-planning and mitigating error propagation across desktop applications. In end-to-end evaluations, IntentCUA achieved a 74.83% task success rate with a Step Efficiency Ratio of 0.91, outperforming RL-based and trajectory-centric baselines. Ablations show that multi-view intent abstraction and shared plan memory jointly improve execution stability, with the cooperative multi-agent loop providing the largest gains on long-horizon tasks. These results highlight that system-level intent abstraction and memory-grounded coordination are key to reliable and efficient desktop automation in large, dynamic environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。