arXiv:2505.18829cs.AIcs.HC2025-05被引 4

让电脑变成语言模型能理解的环境,提升智能体操作效率

LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOS

  • 将电脑转化为语言模型可理解的上下文环境,用MCP服务器抽象操作状态
  • 轻量级智能体LiteCUA在OSWorld上达14.66%成功率,超越复杂框架
  • 适合想构建高效数字系统交互智能体的研究者和开发者

我们提出AIOS 1.0,一个通过环境情境化来提升计算机使用智能体(CUA)能力的新平台。现有方法多聚焦于强化智能体框架或模型,但忽略了语言模型对世界理解与计算机界面结构之间的语义断层。AIOS 1.0通过将计算机转化为语言模型可原生理解的上下文环境,采用模型上下文协议(MCP)服务器架构,抽象计算机状态与操作,有效解耦界面复杂性与决策复杂性,使智能体更高效地推理计算环境。为验证平台效果,我们构建了基于AIOS 1.0的轻量级智能体LiteCUA,其在OSWorld基准测试中取得14.66%的成功率,尽管架构简单,仍优于多个专用智能体框架。结果表明,为语言模型情境化计算机环境是发展更强大计算机使用智能体的有前景方向,有助于实现与数字系统的真正智能交互。

原文摘要 · Abstract (English)

We present AIOS 1.0, a novel platform designed to advance computer-use agent (CUA) capabilities through environmental contextualization. While existing approaches primarily focus on building more powerful agent frameworks or enhancing agent models, we identify a fundamental limitation: the semantic disconnect between how language models understand the world and how computer interfaces are structured. AIOS 1.0 addresses this challenge by transforming computers into contextual environments that language models can natively comprehend, implementing a Model Context Protocol (MCP) server architecture to abstract computer states and actions. This approach effectively decouples interface complexity from decision complexity, enabling agents to reason more effectively about computing environments. To demonstrate our platform's effectiveness, we introduce LiteCUA, a lightweight computer-use agent built on AIOS 1.0 that achieves a 14.66% success rate on the OSWorld benchmark, outperforming several specialized agent frameworks despite its simple architecture. Our results suggest that contextualizing computer environments for language models represents a promising direction for developing more capable computer-use agents and advancing toward AI that can interact with digital systems.

智能体人机交互上下文建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。