用人类协作数据训练双机器人,效率提升5.4倍且性能更优
Duet: Dual-Robot Understanding via Efficient Teaching

- 通过人类协同操作生成数据,再用少量真实机器人轨迹微调
- 相比纯机器人数据训练,任务成功率更高,数据采集效率提升5.4倍
- 适合研究双机器人协作与高效数据收集的科研人员
双机器人协作可完成单个机器人无法实现的任务,如跨环境搬运物体和协调交接。但数据获取是训练这类系统的主要瓶颈。为此,我们提出DUET框架,用于移动操作中的双机器人学习。为高效采集数据,构建了一个统一的、基于同步虚拟现实的异构机器人远程操控系统,用于域内数据收集,并开发了互补跟踪流程,记录人类协作与移动操作先验。为实现高效学习,提出基于动作分块的Transformer架构:先在高效的人类-人类示范数据上预训练协作策略,再在少量真实机器人远程操控轨迹上微调。我们构建了包含四个协作任务的基准测试,使用Unitree G1人形机器人和Dexmate Vega1移动操作臂进行评估。结果表明,利用人类先验不仅显著优于仅使用机器人数据训练的基线模型,还大幅减少人工数据采集负担。我们的数据采集流程平均实现5.4倍加速,且在所有任务上表现相当或更优。
原文摘要 · Abstract (English)
Dual-robot collaboration enables tasks that exceed the reach and payload of a single robot, such as collaboratively transporting objects across environments and executing coordinated handovers. Data acquisition is the primary bottleneck for training these systems. To this end, we introduce DUET, a dual-robot learning framework for mobile manipulation. For efficient data collection, we create a unified dual-embodiment synchronized VR-based teleoperation system for in-domain heterogeneous robot data collection. We further develop a complementary tracking pipeline that records human-human coordination and collaborative mobile manipulation priors. To allow efficient learning, we introduce an Action Chunking Transformer based architecture that first pretrains collaborative policies on efficient human-human demonstrations, before finetuning them on a minimal set of real-robot teleoperation trajectories. We develop a benchmark of four collaborative tasks to evaluate our framework using a Unitree G1 humanoid and a Dexmate Vega1 mobile manipulator. The results demonstrate that harnessing human priors not only yields superior task performance compared to baselines trained only on robot data, but also reduces the total human effort required for data collection. Our human data collection pipeline achieves 5.4x acceleration on average from teleoperation, but we perform equally or better than robot-only data trained policies across all tasks. Our project page is available at https://zhaoy37.github.io/Duet/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。