用视觉与触觉融合提升机器人抓取精度,无需额外调参即可实现实物操作。
NeuralTouch: Neural Descriptors for Precise Sim-to-Real Tactile Robot Control
- 结合神经描述场与触觉反馈,通过强化学习优化抓取姿态。
- 在仿真与真实场景中零样本迁移,抓取成功率显著高于基线方法。
- 适用于复杂接触任务,如插销、开瓶盖,适合高精度操控需求。
抓取精度是精确物体操作的关键前提,常需机器人手与物体间精准对齐。神经描述场(NDF)提供了一种有前景的基于视觉的方法,可生成跨类别物体的泛化抓取姿态。然而,由于相机标定不完善、点云不完整及物体差异性,NDF单独使用可能导致姿态不准。与此同时,触觉感知虽能实现更精确的接触,但现有方法通常仅限于预设的简单接触几何。本文提出NeuralTouch,一种多模态框架,融合NDF与触觉传感,通过轻柔物理交互实现精准、泛化的抓取。该方法利用NDF隐式表示目标接触几何,并在此基础上训练深度强化学习策略,通过触觉反馈精炼抓取,且无需显式指定接触类型。我们在仿真中进行消融实验,并验证其在真实任务(如插销-出孔、开瓶盖)中的零样本迁移能力,无需额外微调。结果表明,NeuralTouch在抓取准确性和鲁棒性上显著优于基线方法,为丰富的接触式机器人操作提供了通用解决方案。
原文摘要 · Abstract (English)
Grasping accuracy is a critical prerequisite for precise object manipulation, often requiring careful alignment between the robot hand and object. Neural Descriptor Fields (NDF) offer a promising vision-based method to generate grasping poses that generalize across object categories. However, NDF alone can produce inaccurate poses due to imperfect camera calibration, incomplete point clouds, and object variability. Meanwhile, tactile sensing enables more precise contact, but existing approaches typically learn policies limited to simple, predefined contact geometries. In this work, we introduce NeuralTouch, a multimodal framework that integrates NDF and tactile sensing to enable accurate, generalizable grasping through gentle physical interaction. Our approach leverages NDF to implicitly represent the target contact geometry, from which a deep reinforcement learning (RL) policy is trained to refine the grasp using tactile feedback. This policy is conditioned on the neural descriptors and does not require explicit specification of contact types. We validate NeuralTouch through ablation studies in simulation and zero-shot transfer to real-world manipulation tasks--such as peg-out-in-hole and bottle lid opening--without additional fine-tuning. Results show that NeuralTouch significantly improves grasping accuracy and robustness over baseline methods, offering a general framework for precise, contact-rich robotic manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。