用普通摄像头视频教便宜机器人学动作，省下大量实拍数据
EasyMimic: A Low-Cost Framework for Robot Imitation Learning from Human Videos
- 从普通视频提取3D手部轨迹，映射到机器人夹爪控制空间
- 仅需少量机器人数据+人类视频，就能快速学会新任务
- 适合家庭机器人研发，低成本、易复现、操作简单
机器人模仿学习常因大规模真实数据采集成本过高而受限，尤其对面向家庭的低成本机器人而言更为突出。为此，我们提出 EasyMimic 框架，一种低成本且可复现的解决方案，使机器人能通过标准RGB摄像头拍摄的人类视频演示，快速学习操控策略。方法首先从视频中提取3D手部轨迹，再通过动作对齐模块将其映射至低成本机器人夹爪控制空间。为弥合人-机域差距，引入简单易用的手部视觉增强策略。随后采用共训练方法，在处理后的人类数据与少量机器人数据上联合微调模型，实现对新任务的快速适应。在低成本 LeRobot 平台上的实验表明，EasyMimic 在多种操作任务中表现优异，显著降低对昂贵机器人数据采集的依赖，为智能机器人进入家庭提供可行路径。
原文摘要 · Abstract (English)
Robot imitation learning is often hindered by the high cost of collecting large-scale, real-world data. This challenge is especially significant for low-cost robots designed for home use, as they must be both user-friendly and affordable. To address this, we propose the EasyMimic framework, a low-cost and replicable solution that enables robots to quickly learn manipulation policies from human video demonstrations captured with standard RGB cameras. Our method first extracts 3D hand trajectories from the videos. An action alignment module then maps these trajectories to the gripper control space of a low-cost robot. To bridge the human-to-robot domain gap, we introduce a simple and user-friendly hand visual augmentation strategy. We then use a co-training method, fine-tuning a model on both the processed human data and a small amount of robot data, enabling rapid adaptation to new tasks. Experiments on the low-cost LeRobot platform demonstrate that EasyMimic achieves high performance across various manipulation tasks. It significantly reduces the reliance on expensive robot data collection, offering a practical path for bringing intelligent robots into homes. Project website: https://zt375356.github.io/EasyMimic-Project/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。