用视觉触觉融合传感器实现高精度滑动检测与材质分类
Robust Slip Detection and Material Classification via Spatiotemporal Transformers on a Uniformly-Illuminated Visuo-Tactile Sensor
- 设计双头时序变换网络处理动态触觉时空数据
- 未见物体上滑动方向识别准确率达91.5%，接触状态预测达95.5%
- 适合需要精细抓握反馈的复杂机器人操作任务
触觉感知是机器人操作的核心，其中滑动检测尤为关键。然而现有滑动数据集多为二分类，缺乏细粒度方向感知。为此，我们提出一种配备定制均匀RGB照明的视觉触觉传感器，构建统一感知框架。硬件层面，传感器实现亚毫米级深度重建；基于此，采集包含15种物体的多任务视觉触觉数据集，每样本同步生成深度信息。算法上，设计双头TimeSformer网络处理动态时空滑动。在未见物体上，该网络在3类接触状态预测中达到95.5%准确率，8类滑动方向分类达91.5%。此外，基于ResNet-50的静态触觉物体分类在15类别上达到98.8%准确率。所提软硬件框架提供高保真反馈，为复杂机器人操作建立强大多模态感知基线。
原文摘要 · Abstract (English)
Tactile sensing is central to robotic manipulation, among which slip detection stands out as a quintessential and critical task. However, existing slip datasets are predominantly limited to binary classification, lacking fine-grained directional perception. To address this limitation, we propose a visuo-tactile sensor featuring customized uniform RGB illumination, alongside a unified perception framework. At the hardware level, the sensor achieves high-precision, sub-millimeter depth reconstruction. Based on this capability, we collect a multi-task visuo-tactile dataset encompassing 15 objects, synchronously generating depth information for each data sample. Algorithmically, we design a dual-head TimeSformer network to process dynamic spatiotemporal slip. On unseen objects, this network achieves robust accuracies of 95.5% and 91.5% for 3-class contact state prediction and fine-grained 8-class slip direction classification, respectively. Furthermore, static tactile-based object class recognition utilizing a ResNet-50 backbone yields an outstanding accuracy of 98.8% across 15 categories. The proposed hardware-software framework provides high-fidelity feedback and a powerful multi-modal perception baseline for complex robotic manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。