通过视觉触觉协同手掌与灵巧手指,实现高精度抓取与操作。
VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation

- 融合视觉与触觉反馈的主动手掌配合可重构手指
- 实现对微小物体(3mm)及易碎物的精准操控
- 适合机器人灵巧操作与学习型算法的数据采集
本文提出一种触觉响应式夹爪,集成视觉-触觉主动手掌(VTAP)与配备触觉阵列传感器的柔性可重构手指。该设计利用手指与手掌的结构协同作用及多模态感知,实现稳定抓取与精细操作。主动双模态手掌无缝结合远距离视觉定位与丰富接触反馈,显著提升系统操作能力。为弥合人手运动与非对称三指结构间的具身差距,进一步提出分阶段、手势条件化的重定向框架,用于灵巧遥操作。大量实验验证了系统在复杂任务中的表现:对YCB及易碎物体的反应式抓取、手持注射器旋转与活塞按压、直径小至3mm的簇状物体分离,以及视觉-触觉引导的插销入孔操作。结果表明,通过协调的手指-手掌交互与多模态感知,无需高自由度仿人结构即可实现高性能操作。VTAP夹爪及其重定向框架为灵巧夹爪设计、操作及接触密集型数据收集提供了实用参考架构,支持基于学习的方法。
原文摘要 · Abstract (English)
This paper presents a tactile-reactive gripper that integrates a Visuo-Tactile Active Palm (VTAP) and compliant, reconfigurable fingers equipped with tactile array sensors. The design exploits structured finger-palm synergy and multi-modal perception to achieve both robust grasping and fine manipulation. The actuated bi-modal palm seamlessly combines long-range visual localization with contact-rich tactile feedback, substantially extending the system's manipulation capability. To bridge the embodiment gap between human hand motion and the heterogeneous three-finger structure, we further propose a staged, gesture-conditioned retargeting framework for dexterous teleoperation. Extensive experiments validate the system across a range of challenging tasks: reactive grasping of YCB and fragile objects, in-hand syringe reorientation and plunger actuation, singulation of clustered objects down to 3 mm in diameter, and vision-tactile peg-in-hole insertion. Results demonstrate that high manipulation performance can be achieved through coordinated finger-palm interaction and multi-modal sensing, without resorting to high degrees of freedom anthropomorphic designs. The VTAP gripper and its retargeting framework offer a practical reference architecture for dexterous gripper design, manipulation, and contact-rich data collection in support of learning-based approaches. Project webpage: https://yuhochau.github.io/vtap/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。