梳理基础模型时代机器人抓取的规划与学习框架。
Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
- 分高阶规划与低阶动作建模,统一学习型方法
- 基础模型生成结构化规划或直接建模可执行动作
- 适合研究机器人操作与多模态学习的学者
视觉、语言和多模态学习的最新进展显著推动了机器人基础模型的发展,而机器人操作仍是最具挑战性的具身任务之一。其难点在于整合感知、语义理解、任务推理、物理驱动的动作生成与可靠执行。本文从算法视角审视机器人操作,通过高阶规划与低阶动作建模的统一抽象,组织近期基于学习的方法。高阶层面,将经典任务规划扩展至涵盖语言、代码、可操作性、几何约束与三维表示的推理;低阶层面,提出面向学习范式的动作模型分类,包括输入建模、潜在空间学习与策略学习。在该抽象框架下,基础模型可通过生成结构化规划结果作为约束或潜在输入,或直接建模可执行动作与轨迹。最后,总结了可扩展性、泛化能力、数据效率、多模态物理交互与安全等开放挑战与未来方向。本综述为机器人操作中基础模型的设计空间与发展趋势提供了系统性视角。
原文摘要 · Abstract (English)
Recent advances in vision, language, and multimodal learning have significantly accelerated progress in robotic foundation models, with robotic manipulation remaining one of the most challenging embodied tasks. Its difficulty lies in integrating perception, semantic understanding, task reasoning, physically grounded action generation, and reliable execution. This survey examines robotic manipulation from an algorithmic perspective and organizes recent learning-based approaches through a unified abstraction of high-level planning and low-level action modeling. At the high level, we extend the classical notion of task planning to include reasoning over language, code, affordances, geometric constraints, and 3D representations. At the low level, we present a learning-paradigm-oriented taxonomy of learning-based action models, covering input modeling, latent learning, and policy learning. Within this abstraction, foundation models contribute either by generating structured planning artifacts that are instantiated as constraints or latent inputs for downstream action generation, or by directly modeling executable actions and trajectories. Finally, we summarize open challenges and future directions related to scalability, generalization, data efficiency, multimodal physical interaction, and safety. Together, this survey provides a structured view of the design space and emerging trends in foundation models for robotic manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。