arXiv:2506.16475cs.ROcs.AI2025-06被引 18

用人类数据训练四足机器人,让其学会灵活抓取物品。

Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining

  • 通过统一人与机器人的观测和动作空间,实现跨体态模仿学习。
  • 在6个真实任务中,成功率平均提升41.9%,分布外情况提升79.7%。
  • 人类预训练显著降低机器人数据需求,适合做机器人操控研究者参考。

四足机器人在复杂环境中已展现卓越的运动能力,但如何以可扩展方式赋予其自主且通用的操纵技能仍是重大挑战。本文提出一种跨体态模仿学习系统,利用人类与洛科曼(LocoMan)四足机器人采集的数据。我们构建了统一且模块化的遥操作与数据采集流程,对齐人与机器人在不同模态下的观测与动作空间。为高效利用数据,设计了一种支持多体态结构化对齐数据联合训练与预训练的模块化架构。此外,首次构建了涵盖多种家庭任务的洛科曼操纵数据集,包含单手与双手模式,并配套对应的人类数据集。我们在6个真实任务上验证系统,整体成功率平均提升41.9%,分布外(OOD)设置下提升79.7%;人类数据预训练使整体成功率提升38.6%,分布外提升82.7%,仅需一半机器人数据即可实现更优性能。代码、硬件与数据已开源。

原文摘要 · Abstract (English)

Quadrupedal robots have demonstrated impressive locomotion capabilities in complex environments, but equipping them with autonomous versatile manipulation skills in a scalable way remains a significant challenge. In this work, we introduce a cross-embodiment imitation learning system for quadrupedal manipulation, leveraging data collected from both humans and LocoMan, a quadruped equipped with multiple manipulation modes. Specifically, we develop a teleoperation and data collection pipeline, which unifies and modularizes the observation and action spaces of the human and the robot. To effectively leverage the collected data, we propose an efficient modularized architecture that supports co-training and pretraining on structured modality-aligned data across different embodiments. Additionally, we construct the first manipulation dataset for the LocoMan robot, covering various household tasks in both unimanual and bimanual modes, supplemented by a corresponding human dataset. We validate our system on six real-world manipulation tasks, where it achieves an average success rate improvement of 41.9% overall and 79.7% under out-of-distribution (OOD) settings compared to the baseline. Pretraining with human data contributes a 38.6% success rate improvement overall and 82.7% under OOD settings, enabling consistently better performance with only half the amount of robot data. Our code, hardware, and data are open-sourced at: https://human2bots.github.io.

四足机器人模仿学习人机协同通用操控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。