用视觉触觉融合提升柔性机械手插入任务的鲁棒性
Visuotactile-Based Learning for Insertion with Compliant Hands
- 结合全景触觉与深度相机，通过迁移学习训练神经策略
- 实机测试中实现无需微调的模拟到现实成功迁移
- 触觉感知对精确定位和稳定插入至关重要，适合复杂装配场景
相比刚性机械手，欠驱动柔性机械手在适应物体形状、实现稳定抓握方面更具优势，且成本更低。然而其固有的柔顺性和缺乏精确指端本体感知，导致手-物交互存在不确定性，尤其在接触密集的插入任务中表现突出。为此需引入额外感知模态以提升鲁棒性。本文探讨了柔性机械手执行插入任务的关键感知需求，聚焦视觉与触觉融合（visuotactile）感知的作用。提出一种基于仿真的多模态策略学习框架，利用全向触觉传感与外置深度相机。采用基于Transformer的策略网络，通过教师-学生蒸馏训练，并成功直接部署至真实机器人系统，无需额外微调。实验表明，触觉与视觉感知协同对于准确估计物-孔位姿、实现可靠的模拟到现实迁移及任务执行具有决定性作用。
原文摘要 · Abstract (English)
Compared to rigid hands, underactuated compliant hands offer greater adaptability to object shapes, provide stable grasps, and are often more cost-effective. However, they introduce uncertainties in hand-object interactions due to their inherent compliance and lack of precise finger proprioception as in rigid hands. These limitations become particularly significant when performing contact-rich tasks like insertion. To address these challenges, additional sensing modalities are required to enable robust insertion capabilities. This letter explores the essential sensing requirements for successful insertion tasks with compliant hands, focusing on the role of visuotactile perception (i.e., visual and tactile perception). We propose a simulation-based multimodal policy learning framework that leverages all-around tactile sensing and an extrinsic depth camera. A transformer-based policy, trained through a teacher-student distillation process, is successfully transferred to a real-world robotic system without further training. Our results emphasize the crucial role of tactile sensing in conjunction with visual perception for accurate object-socket pose estimation, successful sim-to-real transfer and robust task execution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。