arXiv:2510.18316cs.ROcs.AI2025-10被引 14

用约束优化生成多样双臂移动机器人操作数据,仅需少量示范即可训练出可部署的智能策略。

MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation

  • 将数据生成建模为满足硬约束与软约束的优化问题,解决基座位置与视角难题。
  • 在4个任务上生成数据多样性显著提升,单个示范即可训练出成功策略。
  • 仅需40条真实数据微调,就能在真实机器人上成功部署,适合实际应用落地。

从大规模、多样化的真人示范中学习模仿是训练机器人有效的方法,但收集此类数据成本高昂且耗时,尤其在多步骤双臂移动操作任务中,人类需同时操控移动底盘和两个高自由度机械臂。先前的X-Gen方法虽能在仿真中通过少量真人示范生成新场景配置来扩充数据集,但在双臂移动操作任务中面临两大挑战:1)移动底盘引入基座放置问题(影响可达性);2)主动相机引入视角定位问题(影响视觉感知)。为此,MoMaGen将数据生成形式化为一个满足硬约束(如可达性)并平衡软约束(如导航时的可视性)的优化问题。该方法适用于现有大多数自动化数据生成框架,并为未来研究提供理论基础。我们在四个多步骤双臂移动操作任务上进行了评估,结果显示MoMaGen生成的数据集多样性远超以往方法。得益于数据多样性,仅使用单一源示范即可训练出有效的模仿学习策略。此外,该策略只需40条真实世界示范进行微调,即可成功部署于真实机器人硬件。更多细节请见项目页面:momagen.github.io。

原文摘要 · Abstract (English)

Imitation learning from large-scale, diverse human demonstrations has been shown to be effective for training robots, but collecting such data is costly and time-consuming. This challenge intensifies for multi-step bimanual mobile manipulation, where humans must teleoperate both the mobile base and two high-DoF arms. Prior X-Gen works have developed automated data generation frameworks for static (bimanual) manipulation tasks, augmenting a few human demos in simulation with novel scene configurations to synthesize large-scale datasets. However, prior works fall short for bimanual mobile manipulation tasks for two major reasons: 1) a mobile base introduces the problem of how to place the robot base to enable downstream manipulation (reachability) and 2) an active camera introduces the problem of how to position the camera to generate data for a visuomotor policy (visibility). To address these challenges, MoMaGen formulates data generation as a constrained optimization problem that satisfies hard constraints (e.g., reachability) while balancing soft constraints (e.g., visibility while navigation). This formulation generalizes across most existing automated data generation approaches and offers a principled foundation for developing future methods. We evaluate on four multi-step bimanual mobile manipulation tasks and find that MoMaGen enables the generation of much more diverse datasets than previous methods. As a result of the dataset diversity, we also show that the data generated by MoMaGen can be used to train successful imitation learning policies using a single source demo. Furthermore, the trained policy can be fine-tuned with a very small amount of real-world data (40 demos) to be succesfully deployed on real robotic hardware. More details are on our project page: momagen.github.io.

机器人模仿学习数据生成双臂操作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。