探索具身多模态大模型的发展与未来方向
Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directions
- 融合语言、视觉与动作的具身智能模型架构
- 现有数据集支持复杂环境下的感知与交互学习
- 适合关注自主机器人与通用人工智能的研究者
具身多模态大模型（EMLMs）近年来受到广泛关注，因其在复杂真实环境中弥合感知、认知与行动之间差距的潜力。本文全面回顾了该领域的进展，涵盖大语言模型（LLMs）、大视觉模型（LVMs）及其他新兴架构。重点分析了具身感知、导航、交互与仿真能力的演进。文章详细评述了用于训练与评估这些模型的数据集，强调多样且高质量数据对有效学习的重要性。同时指出了当前面临的挑战，包括可扩展性、泛化能力及实时决策问题。最后，提出未来方向：整合多模态感知、推理与动作，推动更自主系统的进步。通过深入分析前沿方法并识别关键空白，本文旨在激发EMLMs在多样化领域中的进一步发展。
原文摘要 · Abstract (English)
Embodied multimodal large models (EMLMs) have gained significant attention in recent years due to their potential to bridge the gap between perception, cognition, and action in complex, real-world environments. This comprehensive review explores the development of such models, including Large Language Models (LLMs), Large Vision Models (LVMs), and other models, while also examining other emerging architectures. We discuss the evolution of EMLMs, with a focus on embodied perception, navigation, interaction, and simulation. Furthermore, the review provides a detailed analysis of the datasets used for training and evaluating these models, highlighting the importance of diverse, high-quality data for effective learning. The paper also identifies key challenges faced by EMLMs, including issues of scalability, generalization, and real-time decision-making. Finally, we outline future directions, emphasizing the integration of multimodal sensing, reasoning, and action to advance the development of increasingly autonomous systems. By providing an in-depth analysis of state-of-the-art methods and identifying critical gaps, this paper aims to inspire future advancements in EMLMs and their applications across diverse domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。