arXiv:2505.06136cs.ROcs.AI2025-05

让机器人用少量示范数据快速学会新任务,适应真实世界。

Efficient Sensorimotor Learning for Open-world Robot Manipulation

  • 基于示范数据中的规律性,实现高效闭环学习。
  • 仅需少量操作演示即可掌握通用抓取技能。
  • 适合构建能持续学习的家用智能机器人。

本论文研究开放世界机器人操作问题,即机器人需在未预编程或预训练的新物体、场景或任务中快速泛化或适应。为此提出高效的感知运动学习方法,核心在于挖掘有限示范数据中的规律性,从而实现数据高效的可泛化操作技能学习。论文提出三个关键贡献:首先,引入以物体为中心的先验知识,使机器人仅通过少量远程操控示范即可学习通用、闭环的感知运动策略;其次,构建机器人的空间理解能力,使其能够从真实环境视频中模仿操作技能;最后,提出从过往经验中识别可复用技能的方法,支持系统连续模仿多个任务。整体贡献为构建低成本数据采集、易与人类交互的通用型个人机器人奠定基础。通过从少量数据中学习与泛化,本工作推动了智能机器人助手在日常场景中无缝集成的愿景。

原文摘要 · Abstract (English)

This dissertation considers Open-world Robot Manipulation, a manipulation problem where a robot must generalize or quickly adapt to new objects, scenes, or tasks for which it has not been pre-programmed or pre-trained. This dissertation tackles the problem using a methodology of efficient sensorimotor learning. The key to enabling efficient sensorimotor learning lies in leveraging regular patterns that exist in limited amounts of demonstration data. These patterns, referred to as ``regularity,'' enable the data-efficient learning of generalizable manipulation skills. This dissertation offers a new perspective on formulating manipulation problems through the lens of regularity. Building upon this notion, we introduce three major contributions. First, we introduce methods that endow robots with object-centric priors, allowing them to learn generalizable, closed-loop sensorimotor policies from a small number of teleoperation demonstrations. Second, we introduce methods that constitute robots' spatial understanding, unlocking their ability to imitate manipulation skills from in-the-wild video observations. Last but not least, we introduce methods that enable robots to identify reusable skills from their past experiences, resulting in systems that can continually imitate multiple tasks in a sequential manner. Altogether, the contributions of this dissertation help lay the groundwork for building general-purpose personal robots that can quickly adapt to new situations or tasks with low-cost data collection and interact easily with humans. By enabling robots to learn and generalize from limited data, this dissertation takes a step toward realizing the vision of intelligent robotic assistants that can be seamlessly integrated into everyday scenarios.

机器人操作少样本学习感知运动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。