用人体触觉数据训练机器人抓握,实现零样本技能迁移。
UniTacHand: Unified Spatio-Tactile Representation for Human to Robotic Hand Skill Transfer
- 将人手与机械手的触觉信号映射到统一的手部3D模型空间。
- 仅用10分钟配对数据训练,即可实现跨域触觉策略迁移。
- 适合做触觉感知、人机技能迁移的研究者和开发者。
触觉感知对机器人实现类人灵巧操作至关重要,尤其在视觉遮挡场景中。然而,大规模真实机器人触觉数据的采集成本高昂。本研究提出利用触觉手套低成本收集人类操作数据,用于基于触觉的机器人策略学习。由于人手与机器人触觉数据存在结构差异,导致策略迁移困难。为此,我们提出UniTacHand,一种统一表示方法,将灵巧手捕捉的机器人触觉信息与手套获取的人类触觉对齐。首先,将人手与机器人触觉信号投影至形态一致的MANO手模型二维表面空间,标准化异构数据并嵌入空间上下文。随后,引入对比学习方法,在仅10分钟配对数据下将其对齐至统一潜在空间。该方法实现从人类到真实机器人的零样本触觉策略迁移,并可泛化至预训练中未见物体。此外,通过UniTacHand联合训练混合数据(含人类与机器人示范),相比仅使用机器人数据,性能更优且数据效率更高。UniTacHand为触觉驱动灵巧手的通用、可扩展、高效学习提供了新路径。
原文摘要 · Abstract (English)
Tactile sensing is crucial for robotic hands to achieve human-level dexterous manipulation, especially in scenarios with visual occlusion. However, its application is often hindered by the difficulty of collecting large-scale real-world robotic tactile data. In this study, we propose to collect low-cost human manipulation data using haptic gloves for tactile-based robotic policy learning. The misalignment between human and robotic tactile data makes it challenging to transfer policies learned from human data to robots. To bridge this gap, we propose UniTacHand, a unified representation to align robotic tactile information captured by dexterous hands with human hand touch obtained from gloves. First, we project tactile signals from both human hands and robotic hands onto a morphologically consistent 2D surface space of the MANO hand model. This unification standardizes the heterogeneous data structures and inherently embeds the tactile signals with spatial context. Then, we introduce a contrastive learning method to align them into a unified latent space, trained on only 10 minutes of paired data from our data collection system. Our approach enables zero-shot tactile-based policy transfer from humans to a real robot, generalizing to objects unseen in the pre-training data. We also demonstrate that co-training on mixed data, including both human and robotic demonstrations via UniTacHand, yields better performance and data efficiency compared with using only robotic data. UniTacHand paves a path toward general, scalable, and data-efficient learning for tactile-based dexterous hands.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。