HumDex让机器人灵巧操作更简单,只需一套便携设备就能高效收集高质量人类动作数据。
HumDex: Humanoid Dexterous Manipulation Made Easy
- 用惯性传感器实现全身高精度、易部署的动作捕捉
- 学习式手部动作重定向,无需调参即可生成自然手势
- 两阶段模仿学习提升泛化能力,少量数据即可适应新物体与环境
本文研究人形机器人全身灵巧操作,其高效获取高质量示范数据仍是核心瓶颈。现有遥操作系统常受限于可移动性差、遮挡或精度不足,难以适用于复杂全身任务。为此,我们提出HumDex,一种面向人形机器人全身灵巧操作的便携式遥操作系统。系统采用基于惯性测量单元(IMU)的动作追踪技术,在保证高精度的同时实现便捷部署。针对灵巧手控制,进一步引入基于学习的重定向方法,无需人工调参即可生成平滑自然的手部运动。除遥操作外,HumDex支持高效的人类动作数据采集。基于此能力,我们提出两阶段模仿学习框架:先在多样人类动作数据上预训练以学习通用先验,再在机器人数据上微调以弥合具身差距。实验表明,该方法显著提升了对新配置、新物体和新背景的泛化能力,且数据采集成本极低。整个系统完全可复现,并开源于https://github.com/physical-superintelligence-lab/humdex。
原文摘要 · Abstract (English)
This paper investigates humanoid whole-body dexterous manipulation, where the efficient collection of high-quality demonstration data remains a central bottleneck. Existing teleoperation systems often suffer from limited portability, occlusion, or insufficient precision, which hinders their applicability to complex whole-body tasks. To address these challenges, we introduce HumDex, a portable teleoperation system designed for humanoid whole-body dexterous manipulation. Our system leverages IMU-based motion tracking to address the portability-precision trade-off, enabling accurate full-body tracking while remaining easy to deploy. For dexterous hand control, we further introduce a learning-based retargeting method that generates smooth and natural hand motions without manual parameter tuning. Beyond teleoperation, HumDex enables efficient collection of human motion data. Building on this capability, we propose a two-stage imitation learning framework that first pre-trains on diverse human motion data to learn generalizable priors, and then fine-tunes on robot data to bridge the embodiment gap for precise execution. We demonstrate that this approach significantly improves generalization to new configurations, objects, and backgrounds with minimal data acquisition costs. The entire system is fully reproducible and open-sourced at https://github.com/physical-superintelligence-lab/humdex.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。