arXiv:2510.10991cs.CVcs.AI2025-10综述被引 19

系统梳理具智能体特性的多模态大模型,揭示其自主决策能力。

A Survey on Agentic Multimodal Large Language Models

  • 从三维度构建智能体框架:内部推理记忆、外部工具调用、环境交互行为
  • 提出可扩展的评估与训练资源库,支持研究快速迭代
  • 适合关注AI自主性与通用智能的研究者和开发者

随着自主智能体系统的兴起,研究界正经历从传统静态、被动、领域专用AI代理向更动态、主动、泛化的智能体AI的重大转变。为响应对智能体AI日益增长的兴趣及其向通用人工智能(AGI)发展的潜力,本文全面综述了具智能体特性的多模态大语言模型(Agentic MLLMs)。我们探讨了该新兴范式,厘清其概念基础,并区分其与传统基于MLLM的代理的不同特征。本文建立了一个包含三个核心维度的概念框架:(i) 内部智能功能作为系统指挥官,通过推理、反思和记忆实现长程规划;(ii) 外部工具调用,使模型能主动使用各类外部工具以拓展其问题求解能力;(iii) 环境交互,将模型置于虚拟或物理环境中,使其能够采取行动、调整策略,并在动态真实场景中持续保持目标导向行为。为加速该领域研究,我们整理了开源训练框架及训练与评估数据集。最后,我们回顾了其下游应用并展望未来研究方向。为持续追踪该快速演进领域的进展,我们将在 https://github.com/HJYao00/Awesome-Agentic-MLLMs 持续更新公共资源库。

原文摘要 · Abstract (English)

With the recent emergence of revolutionary autonomous agentic systems, research community is witnessing a significant shift from traditional static, passive, and domain-specific AI agents toward more dynamic, proactive, and generalizable agentic AI. Motivated by the growing interest in agentic AI and its potential trajectory toward AGI, we present a comprehensive survey on Agentic Multimodal Large Language Models (Agentic MLLMs). In this survey, we explore the emerging paradigm of agentic MLLMs, delineating their conceptual foundations and distinguishing characteristics from conventional MLLM-based agents. We establish a conceptual framework that organizes agentic MLLMs along three fundamental dimensions: (i) Agentic internal intelligence functions as the system's commander, enabling accurate long-horizon planning through reasoning, reflection, and memory; (ii) Agentic external tool invocation, whereby models proactively use various external tools to extend their problem-solving capabilities beyond their intrinsic knowledge; and (iii) Agentic environment interaction further situates models within virtual or physical environments, allowing them to take actions, adapt strategies, and sustain goal-directed behavior in dynamic real-world scenarios. To further accelerate research in this area for the community, we compile open-source training frameworks, training and evaluation datasets for developing agentic MLLMs. Finally, we review the downstream applications of agentic MLLMs and outline future research directions for this rapidly evolving field. To continuously track developments in this rapidly evolving field, we will also actively update a public repository at https://github.com/HJYao00/Awesome-Agentic-MLLMs.

智能体多模态大模型AGI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。