系统梳理机器人领域基础模型的发展脉络与关键技术。
Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions

- 按五阶段演进梳理机器人基础模型发展路径
- 全面对比多模态模型类型与学习范式差异
- 适合关注机器人智能化升级的研究者阅读
近年来,机器人领域正从固定、单一任务的专用方案,转向具备自适应、多功能和通用能力的智能体,能够在复杂、开放世界和动态环境中运行。这一变革主要由基础模型(Foundation Models, FMs)推动,即在大规模异构数据上训练的大规模神经网络架构,具备前所未有的多模态理解与推理、长时程规划及跨体感泛化能力。本文对机器人领域的基础模型研究进行了全面、系统、深入的综述。首先,通过五个不同研究阶段,勾勒出该领域的发展脉络,涵盖从早期引入自然语言处理(NLP)与计算机视觉(CV)模型,到当前多感官泛化与真实世界部署的前沿进展。随后,从模型类型(如大语言模型、视觉-语言模型、视觉-语言动作模型)、神经网络架构、学习范式、知识融入阶段、主要机器人任务及实际应用领域六个维度,进行精细化分类分析,并提供比较研究与批判性见解。同时,整理了用于模型训练与评估的公开数据集。最后,从层级角度探讨当前面临的挑战与未来有前景的研究方向。
原文摘要 · Abstract (English)
Over the recent years, the field of robotics has been undergoing a transformative paradigm shift from fixed, single-task, domain-specific solutions towards adaptive, multi-function, generalpurpose agents, capable of operating in complex, open-world, and dynamic environments. This tremendous advancement is primarily driven by the emergence of Foundation Models (FMs), i.e., large-scale neural-network architectures trained on massive, heterogeneous datasets that provide unprecedented capabilities in multi-modal understanding and reasoning, long-horizon planning, and cross-embodiment generalization. In this context, the current study provides a holistic, systematic, and in-depth review of the research landscape of FMs in robotics. In particular, the evolution of the field is initially delineated through five distinct research phases, spanning from the early incorporation of Natural Language Processing (NLP) and Computer Vision (CV) models to the current frontier of multi-sensory generalization and real-world deployment. Subsequently, a highly-granular taxonomic investigation of the literature is performed, examining the following key aspects: a) the employed FM types, including LLMs, VFMs, VLMs, and VLAs, b) the underlying neural-network architectures, c) the adopted learning paradigms, d) the different learning stages of knowledge incorporation, e) the major robotic tasks, and f) the main real-world application domains. For each aspect, comparative analysis and critical insights are provided. Moreover, a report on the publicly available datasets used for model training and evaluation across the considered robotic tasks is included. Furthermore, a hierarchical discussion on the current open challenges and promising future research directions in the field is incorporated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。