arXiv:2409.03403cs.RO2024-09CoRL被引 78

用生成模型合成不同机器人和视角数据,提升机器人技能跨体感迁移效果。

RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learning

  • 通过图像生成技术合成多机器人、多视角的演示数据。
  • 在未见过的机器人上实现零样本部署,成功率最高提升30%。
  • 无需测试时调整,适合需要快速迁移的多机器人系统。

扩大机器人学习需大规模多样数据集,如何高效复用数据并跨体感迁移策略仍是挑战。尽管如Open-X Embodiment(OXE)项目已展示技能整合潜力,但数据集中机器人类型与相机视角分布不均仍导致策略过拟合。为此,本文提出RoVi-Aug,利用先进图像到图像生成模型,合成不同机器人与相机视角下的演示数据,以增强训练。通过大量物理实验验证,基于机器人与视角增广数据训练的策略可零样本部署于视角差异显著的新机器人上。相比测试时自适应方法Mirage,RoVi-Aug无需测试时额外处理,不依赖已知相机参数,且支持策略微调。通过联合训练原始与增广数据集,可学习多机器人、多任务策略,实现更高效的体感与技能迁移,成功率最高提升30%。项目主页:https://rovi-aug.github.io。

原文摘要 · Abstract (English)

Scaling up robot learning requires large and diverse datasets, and how to efficiently reuse collected data and transfer policies to new embodiments remains an open question. Emerging research such as the Open-X Embodiment (OXE) project has shown promise in leveraging skills by combining datasets including different robots. However, imbalances in the distribution of robot types and camera angles in many datasets make policies prone to overfit. To mitigate this issue, we propose RoVi-Aug, which leverages state-of-the-art image-to-image generative models to augment robot data by synthesizing demonstrations with different robots and camera views. Through extensive physical experiments, we show that, by training on robot- and viewpoint-augmented data, RoVi-Aug can zero-shot deploy on an unseen robot with significantly different camera angles. Compared to test-time adaptation algorithms such as Mirage, RoVi-Aug requires no extra processing at test time, does not assume known camera angles, and allows policy fine-tuning. Moreover, by co-training on both the original and augmented robot datasets, RoVi-Aug can learn multi-robot and multi-task policies, enabling more efficient transfer between robots and skills and improving success rates by up to 30%. Project website: https://rovi-aug.github.io.

机器人学习数据增强跨体感迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。