arXiv:2508.04700cs.AIcs.CL2025-08被引 60

让电脑助手自己学用新软件,无需人工标注。

SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience

  • 通过试错积累经验,逐步掌握陌生软件操作。
  • 在五个新环境中成功率从11.3%提升至34.5%,提高23.2%。
  • 适合想构建能持续自学的智能助手的研究者与开发者。

将大视觉语言模型(LVLM)重用于计算机使用代理(CUA)已取得显著进展,主要依赖人工标注数据。然而,这些模型在缺乏标注的新颖和专用软件场景中表现不佳。为此,我们提出SEAgent,一种通过与未知软件交互实现自主进化的代理框架。SEAgent使计算机使用代理能够通过经验学习,自主探索新软件,经历迭代试错,并逐步完成由简单到复杂的自动生成任务。为此,我们设计了分步轨迹评估的世界状态模型,以及生成日益多样化且更具挑战性的任务的课程生成器。代理策略通过经验学习更新,包括对失败动作的对抗性模仿和对成功动作的组相对策略优化(GRPO)。此外,我们引入专家到通用的训练策略,整合多个专家代理的经验洞察,促进更强通用代理的形成,使其具备持续自主进化能力。该统一代理在专有软件上的性能超越了独立专家代理的集成。我们在OS-World中的五个新型软件环境中验证了SEAgent的有效性,相较于开源基准模型UI-TARS,成功率提升了23.2%(从11.3%到34.5%)。

原文摘要 · Abstract (English)

Repurposing large vision-language models (LVLMs) as computer use agents (CUAs) has led to substantial breakthroughs, primarily driven by human-labeled data. However, these models often struggle with novel and specialized software, particularly in scenarios lacking human annotations. To address this challenge, we propose SEAgent, an agentic self-evolving framework enabling CUAs to autonomously evolve through interactions with unfamiliar software. Specifically, SEAgent empowers computer-use agents to autonomously master novel software environments via experiential learning, where agents explore new software, learn through iterative trial-and-error, and progressively tackle auto-generated tasks organized from simple to complex. To achieve this goal, we design a World State Model for step-wise trajectory assessment, along with a Curriculum Generator that generates increasingly diverse and challenging tasks. The agent's policy is updated through experiential learning, comprised of adversarial imitation of failure actions and Group Relative Policy Optimization (GRPO) on successful ones. Furthermore, we introduce a specialist-to-generalist training strategy that integrates individual experiential insights from specialist agents, facilitating the development of a stronger generalist CUA capable of continuous autonomous evolution. This unified agent ultimately achieves performance surpassing ensembles of individual specialist agents on their specialized software. We validate the effectiveness of SEAgent across five novel software environments within OS-World. Our approach achieves a significant improvement of 23.2% in success rate, from 11.3% to 34.5%, over a competitive open-source CUA, i.e., UI-TARS.

AI代理自主学习软件操作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。