arXiv:2510.19386cs.MAcs.AI2025-10被引 4

ColorAgent让操作系统像伙伴一样主动理解用户并长期可靠执行任务。

ColorAgent: Building A Robust, Personalized, and Interactive OS Agent

  • 用分步强化学习和自进化训练提升模型长期交互能力
  • 在安卓环境测试中成功率达77.2%(AndroidWorld)和50.7%(AndroidLab)
  • 支持个性化意图识别,可主动协作,适合智能助手研发者参考

随着硬件、软件及大语言模型技术的进步,人机交互已从命令行发展为新兴的AI代理模式。本文提出ColorAgent,一个具备长周期、鲁棒性环境交互能力,并支持个性化与主动性用户互动的操作系统代理。为实现长期交互,采用分步强化学习与自进化训练,构建定制化多代理框架以保障通用性、一致性和鲁棒性。在用户交互方面,探索个性化意图识别与主动参与机制,使代理不仅是自动化工具,更成为温暖的合作伙伴。在AndroidWorld与AndroidLab基准测试中,成功率分别达到77.2%和50.7%,刷新当前最佳表现。然而,现有基准仍不足以全面评估操作系统代理,未来需在评估范式、代理协作与安全性等方面进一步探索。

原文摘要 · Abstract (English)

With the advancements in hardware, software, and large language model technologies, the interaction between humans and operating systems has evolved from the command-line interface to the rapidly emerging AI agent interactions. Building an operating system (OS) agent capable of executing user instructions and faithfully following user desires is becoming a reality. In this technical report, we present ColorAgent, an OS agent designed to engage in long-horizon, robust interactions with the environment while also enabling personalized and proactive user interaction. To enable long-horizon interactions with the environment, we enhance the model's capabilities through step-wise reinforcement learning and self-evolving training, while also developing a tailored multi-agent framework that ensures generality, consistency, and robustness. In terms of user interaction, we explore personalized user intent recognition and proactive engagement, positioning the OS agent not merely as an automation tool but as a warm, collaborative partner. We evaluate ColorAgent on the AndroidWorld and AndroidLab benchmarks, achieving success rates of 77.2% and 50.7%, respectively, establishing a new state of the art. Nonetheless, we note that current benchmarks are insufficient for a comprehensive evaluation of OS agents and propose further exploring directions in future work, particularly in the areas of evaluation paradigms, agent collaboration, and security.

操作系统代理长周期交互个性化多智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。