arXiv:2412.10047cs.AI2024-12被引 36

从语言模型到行动模型,构建可执行真实任务的智能体系统

Large Action Models: From Inception to Implementation

  • 以操作系统为环境,分步骤构建可执行任务的行动模型
  • 提出从数据收集到评估的全流程框架,支持多场景部署
  • 适合研究智能体、具身AI及实际应用落地的开发者

随着人工智能的发展,对超越语言辅助、具备现实世界行动能力的智能体的需求日益增长。这一演进要求从擅长生成文本的传统大语言模型(LLMs)转向专为动态环境中动作生成与执行设计的大行动模型(LAMs)。依托智能体系统,LAMs有望将AI从被动的语言理解转变为主动的任务完成,是迈向通用人工智能的重要一步。本文提出一个完整的LAM开发框架,系统性地涵盖从概念到部署的全过程。以基于Windows OS的智能体为例,详细介绍了数据收集、模型训练、环境集成、状态锚定和评估等关键阶段。该通用工作流可作为多个应用场景下构建功能性LAMs的蓝图。最后,我们指出当前LAMs的局限性,并探讨未来研究与产业落地的方向,强调实现其在真实世界中潜力所面临的挑战与机遇。本文使用的数据收集代码已公开于:https://github.com/microsoft/UFO/tree/main/dataflow,完整文档见:https://microsoft.github.io/UFO/dataflow/overview/。

原文摘要 · Abstract (English)

As AI continues to advance, there is a growing demand for systems that go beyond language-based assistance and move toward intelligent agents capable of performing real-world actions. This evolution requires the transition from traditional Large Language Models (LLMs), which excel at generating textual responses, to Large Action Models (LAMs), designed for action generation and execution within dynamic environments. Enabled by agent systems, LAMs hold the potential to transform AI from passive language understanding to active task completion, marking a significant milestone in the progression toward artificial general intelligence. In this paper, we present a comprehensive framework for developing LAMs, offering a systematic approach to their creation, from inception to deployment. We begin with an overview of LAMs, highlighting their unique characteristics and delineating their differences from LLMs. Using a Windows OS-based agent as a case study, we provide a detailed, step-by-step guide on the key stages of LAM development, including data collection, model training, environment integration, grounding, and evaluation. This generalizable workflow can serve as a blueprint for creating functional LAMs in various application domains. We conclude by identifying the current limitations of LAMs and discussing directions for future research and industrial deployment, emphasizing the challenges and opportunities that lie ahead in realizing the full potential of LAMs in real-world applications. The code for the data collection process utilized in this paper is publicly available at: https://github.com/microsoft/UFO/tree/main/dataflow, and comprehensive documentation can be found at https://microsoft.github.io/UFO/dataflow/overview/.

行动模型智能体AI落地

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。