用单臂机器人+人类协作,低成本训练双臂机械臂操作策略
MonoDuo: Using One Robot Arm to Learn Bimanual Policies

- 通过人机协同采集单臂机器人数据,合成双臂任务演示
- 零样本部署在未见过的双臂机器人上,成功率最高达70%
- 仅需25个目标机器人示例即可显著提升性能,适合缺乏双臂机器人的研究者
双臂协调对众多实际操作任务至关重要,但双臂机器人和数据集稀缺限制了相关策略的学习。单臂机器人在实验室中广泛存在,能否利用它们训练双臂策略?我们提出MonoDuo框架,通过单臂机器人示范与人类协作收集数据:一人操控单臂完成任务一侧,另一人完成另一侧,再交换角色覆盖双向。使用腕部和固定摄像头获取的RGB-D观测,结合先进的手部姿态估计、图像与点云分割及修复技术,生成基于真实机器人运动学的合成演示。这些合成数据用于训练双臂策略。我们在五项任务(箱体搬运、背包打包、布料折叠、夹克拉链、盘子交接)上评估,相比仅依赖人类双臂视频的方法,MonoDuo实现对未见双臂机器人配置的零样本部署,成功率达70%;仅需25个目标机器人示范,少样本微调使成功率比从零训练提升65%-70%,证明该方法能高效将单臂数据知识迁移至双臂策略。
原文摘要 · Abstract (English)
Bimanual coordination is essential for many real-world manipulation tasks, yet learning bimanual robot policies is limited by the scarcity of bimanual robots and datasets. Single-arm robots, however, are widely available in research labs. Can we leverage them to train bimanual robot policies? We present MonoDuo, a framework for learning bimanual manipulation policies using single-arm robot demonstrations paired with human collaboration. MonoDuo collects data by teleoperating a single-arm robot to perform one side of a bimanual task while a human performs the other, then swapping roles to cover both sides. RGB-D observations from a wrist-mounted and fixed camera are augmented into synthetic demonstrations for target bimanual robots using state-of-the-art hand pose estimation, image and point cloud segmentation, and inpainting. These synthetic demonstrations, grounded in real robot kinematics, are used to train bimanual policies. We evaluate MonoDuo on five tasks: box lifting, backpack packing, cloth folding, jacket zipping, and plate handover. Compared to approaches relying solely on human bimanual videos, MonoDuo enables zero-shot deployment on unseen bimanual robot configurations, achieving success rates up to 70%. With only 25 target robot demonstrations, few-shot finetuning further boosts success rates by 65-70% over training from scratch, demonstrating MonoDuo's effectiveness in efficiently transferring knowledge from single-arm robot data to bimanual robot policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。