用新表征和力感知预训练提升机器人触觉抓取能力
Canonical Representation and Force-Based Pretraining of 3D Tactile for Dexterous Visuo-Tactile Policy Learning
- 设计统一坐标系下的3D触觉表征，降低高维数据学习难度
- 通过力信号自监督预训练，同时捕捉局部与整体受力特征
- 在4个真实场景任务中平均成功率78%，适合精密操作研究
触觉传感对机器人执行精细、接触密集的任务至关重要。然而，由于灵巧手覆盖面积大，触觉数据维度极高，给有效特征学习带来挑战，尤其在3D触觉数据领域，缺乏大规模标准化数据集和强预训练主干网络。为此，我们提出一种新的规范表征，降低3D触觉特征学习难度，并引入基于力的自监督预训练任务，以捕捉对灵巧操作至关重要的局部与净力特征。该方法在真实世界实验中，在四个精细、接触密集的灵巧操作任务上达到平均78%的成功率，展现出显著的有效性和鲁棒性。进一步分析表明，该方法充分融合了3D触觉数据中的空间与力信息以完成任务。代码与视频见https://3dtacdex.github.io。
原文摘要 · Abstract (English)
Tactile sensing plays a vital role in enabling robots to perform fine-grained, contact-rich tasks. However, the high dimensionality of tactile data, due to the large coverage on dexterous hands, poses significant challenges for effective tactile feature learning, especially for 3D tactile data, as there are no large standardized datasets and no strong pretrained backbones. To address these challenges, we propose a novel canonical representation that reduces the difficulty of 3D tactile feature learning and further introduces a force-based self-supervised pretraining task to capture both local and net force features, which are crucial for dexterous manipulation. Our method achieves an average success rate of 78% across four fine-grained, contact-rich dexterous manipulation tasks in real-world experiments, demonstrating effectiveness and robustness compared to other methods. Further analysis shows that our method fully utilizes both spatial and force information from 3D tactile data to accomplish the tasks. The codes and videos can be viewed at https://3dtacdex.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。