arXiv:2506.15847cs.ROcs.AI2025-06被引 10

看一眼人类操作视频,机器人就能安全自主学会复杂家务动作。

SafeMimic: Towards Safe and Autonomous Human-to-Robot Imitation for Mobile Manipulation

  • 从单个第三人称视频中解析任务步骤与动作,转换为机器人自身视角
  • 通过模拟训练的安全评估机制,确保执行过程零碰撞,失败时自动回溯调整
  • 适用于不同用户、环境和机器人形态,显著减少后续探索尝试次数

为了让机器人成为家庭中的高效助手,它们必须能仅通过观看人类演示就学会新的移动操作任务。从单个第三人称人类视频中学习极具挑战性:机器人需先从示范中提取任务目标与执行方式,将策略从第三人称转换为第一人称视角,并适配自身的身体结构。此外,为降低对昂贵人工监控的依赖,整个学习过程必须安全且自主完成。本文提出 SafeMimic 框架,实现从单一人类视频中安全、自主地学习多步移动操作技能。给定一个多人多步骤任务的初始视频示范,SafeMimic 首先将视频分段,推断语义变化及对应的人类动作,并将其转换为机器人自身的视角。随后,通过在人类动作附近采样候选动作,并利用模拟训练的集成安全Q函数进行安全验证,在递推视野内决定是否执行。若无法安全推进,则回溯至先前状态并尝试不同动作序列,必要时同时调整轨迹与抓取模式。结果表明,该方法可在不同用户、环境和机器人形态下,成功学习多步移动操作行为,相较现有最优基线在七个任务上均取得提升。

原文摘要 · Abstract (English)

For robots to become efficient helpers in the home, they must learn to perform new mobile manipulation tasks simply by watching humans perform them. Learning from a single video demonstration from a human is challenging as the robot needs to first extract from the demo what needs to be done and how, translate the strategy from a third to a first-person perspective, and then adapt it to be successful with its own morphology. Furthermore, to mitigate the dependency on costly human monitoring, this learning process should be performed in a safe and autonomous manner. We present SafeMimic, a framework to learn new mobile manipulation skills safely and autonomously from a single third-person human video. Given an initial human video demonstration of a multi-step mobile manipulation task, SafeMimic first parses the video into segments, inferring both the semantic changes caused and the motions the human executed to achieve them and translating them to an egocentric reference. Then, it adapts the behavior to the robot's own morphology by sampling candidate actions around the human ones, and verifying them for safety before execution in a receding horizon fashion using an ensemble of safety Q-functions trained in simulation. When safe forward progression is not possible, SafeMimic backtracks to previous states and attempts a different sequence of actions, adapting both the trajectory and the grasping modes when required for its morphology. As a result, SafeMimic yields a strategy that succeeds in the demonstrated behavior and learns task-specific actions that reduce exploration in future attempts. Our experiments show that our method allows robots to safely and efficiently learn multi-step mobile manipulation behaviors from a single human demonstration, from different users, and in different environments, with improvements over state-of-the-art baselines across seven tasks

机器人模仿安全学习视觉理解移动操作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。