用真人手部动作采集数据，让机器人在陌生环境更灵活操作。
DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies
- 让人用手在真实环境中交互，低成本收集多样化操作数据
- 新环境成功率68.5%，是仅用机器人数据训练的4倍
- 适合需要跨场景、跨机械臂泛化的机器人研究者
大规模、多样化的机器人数据集为实现灵巧操作策略在新环境中的泛化提供了新路径，但数据采集面临诸多挑战。尽管遥操作能提供高质量数据，但成本高昂限制了其可扩展性。本文提出，能否让人们像日常生活中一样，直接用自己的手来采集数据？在DexWild中，一组多样化数据采集者使用双手，在多种真实环境中与各类物体进行交互，积累了数小时的数据。为此，我们开发了低成本、便携且易用的DexWild-System设备用于记录。DexWild学习框架同时利用人类和机器人示范数据进行联合训练，相比单独使用任一数据集，性能显著提升。该方法生成的机器人策略能有效泛化到新环境、新任务及新机械结构，仅需少量机器人专属数据即可。实验表明，DexWild在未见过的环境中达到68.5%的成功率，近乎仅用机器人数据训练的四倍；跨机械结构泛化能力提升5.8倍。视频演示、代码及使用说明见https://dexwild.github.io
原文摘要 · Abstract (English)
Large-scale, diverse robot datasets have emerged as a promising path toward enabling dexterous manipulation policies to generalize to novel environments, but acquiring such datasets presents many challenges. While teleoperation provides high-fidelity datasets, its high cost limits its scalability. Instead, what if people could use their own hands, just as they do in everyday life, to collect data? In DexWild, a diverse team of data collectors uses their hands to collect hours of interactions across a multitude of environments and objects. To record this data, we create DexWild-System, a low-cost, mobile, and easy-to-use device. The DexWild learning framework co-trains on both human and robot demonstrations, leading to improved performance compared to training on each dataset individually. This combination results in robust robot policies capable of generalizing to novel environments, tasks, and embodiments with minimal additional robot-specific data. Experimental results demonstrate that DexWild significantly improves performance, achieving a 68.5% success rate in unseen environments-nearly four times higher than policies trained with robot data only-and offering 5.8x better cross-embodiment generalization. Video results, codebases, and instructions at https://dexwild.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。