arXiv:2502.16796cs.MAcs.AI2025-02KDD被引 24

让手机助手自动完成跨应用任务,还能自我改进。

MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions

  • 用多个专用代理+中央协调器,分工协作处理跨应用任务。
  • 在真实环境测试中,准确率显著优于现有单/多代理方案。
  • 适合想自动化复杂手机操作的用户或研究智能助手的开发者。

手机智能体可帮助用户自动化日常手机任务,已成为研究热点。然而,现有以流程为导向的智能体在处理跨应用指令时面临三大挑战:(1)任务关系复杂,(2)应用环境多样,(3)多步执行中的错误传播与信息丢失。受面向对象编程启发,我们提出自进化多智能体框架MobileSteward,集成多个面向应用的StaffAgent,由中央协调的StewardAgent统一调度。框架包含三个模块:(1)动态招募生成基于信息流的调度图,显式关联跨应用任务;(2)任务指派将任务分配给具备应用专长的StaffAgent,应对应用差异;(3)调整评估通过反馈提示或关键信息传递,缓解错误传播与信息丢失。为持续提升性能,我们设计基于记忆的自进化机制,总结成功执行经验。我们构建了首个真实环境下英文跨应用基准(CAPBench)用于评估。实验表明,MobileSteward在复杂跨应用指令处理上优于单/多智能体框架,展现出更强的适应性与鲁棒性。

原文摘要 · Abstract (English)

Mobile phone agents can assist people in automating daily tasks on their phones, which have emerged as a pivotal research spotlight. However, existing procedure-oriented agents struggle with cross-app instructions, due to the following challenges: (1) complex task relationships, (2) diverse app environment, and (3) error propagation and information loss in multi-step execution. Drawing inspiration from object-oriented programming principles, we recognize that object-oriented solutions is more suitable for cross-app instruction. To address these challenges, we propose a self-evolving multi-agent framework named MobileSteward, which integrates multiple app-oriented StaffAgents coordinated by a centralized StewardAgent. We design three specialized modules in MobileSteward: (1) Dynamic Recruitment generates a scheduling graph guided by information flow to explicitly associate tasks among apps. (2) Assigned Execution assigns the task to app-oriented StaffAgents, each equipped with app-specialized expertise to address the diversity between apps. (3) Adjusted Evaluation conducts evaluation to provide reflection tips or deliver key information, which alleviates error propagation and information loss during multi-step execution. To continuously improve the performance of MobileSteward, we develop a Memory-based Self-evolution mechanism, which summarizes the experience from successful execution, to improve the performance of MobileSteward. We establish the first English Cross-APP Benchmark (CAPBench) in the real-world environment to evaluate the agents' capabilities of solving complex cross-app instructions. Experimental results demonstrate that MobileSteward achieves the best performance compared to both single-agent and multi-agent frameworks, highlighting the superiority of MobileSteward in better handling user instructions with diverse complexity.

手机智能体多智能体自进化跨应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。