arXiv:2608.15490cs.RO2026-08

用视觉捕捉触觉变形,让机器人更懂摸东西

Vision-Based Tactile Intelligence for Robotics: Sensing, Learning, and Embodied Manipulation

论文配图:Vision-Based Tactile Intelligence for Robotics: Sensing, Learning, and Embodied Manipulation
图 1 · 摘自论文原文
  • 通过软体变形成像实现高分辨率触觉感知
  • 构建从信号理解到任务策略的分层学习体系
  • 整合仿真与数据集支持跨传感器迁移应用

触觉感知对接触密集型机器人任务至关重要,但许多触觉传感器仍提供稀疏、低维信号,难以支撑复杂感知与交互。基于视觉的触觉传感器(VBTS)通过将软界面受力变形转换为图像,提供高分辨率、信息丰富的触觉观测,支持复杂机器人任务。本文综述了完整的VBTS技术链,将传感硬件、学习方法、仿真平台与数据集视为一体化的感知-学习系统:1)按可变形弹性体设计、传感器尺寸形状及光学系统设计,建立硬件分类体系,指导未来传感器开发;2)构建从底层信号理解到任务级策略与基础模型的分层学习框架;3)分析仿真平台与触觉数据集作为规模化训练与部署的支撑层,涵盖模拟到现实迁移及跨传感器适应。最后指出当前挑战与未来方向。本文通过揭示硬件、人工智能架构、仿真与数据集间的协同机制,推动接触密集型机器人任务中的触觉智能发展。

原文摘要 · Abstract (English)

Tactile sensing is essential for robots in contact-rich tasks, yet many tactile sensors still provide sparse, low-dimensional signals that do not capture sufficient information for complex robotic perception and interaction. Vision-based tactile sensors (VBTSs) offer a powerful alternative by con-verting contact-induced deformation of a soft interface into im-ages. The image-based formulation gives VBTSs high-resolution, information-rich tactile observations that enable complex robotic tasks. This review surveys the full VBTS pipeline and treats sensing hardware, learning methods, simulation, and datasets as an integrated sensing-and-learning system. We 1) organize representative VBTSs into a hardware taxonomy structured by deformable elastomer design, sensor size and shape, and optical system design to guide future sensor development; 2) present a hierarchical view of learning-based tactile intelligence from low-level signal understanding to task-level policies and foundation models; and 3) examine simulation platforms and tactile datasets as a scaling layer, together with sim-to-real transfer and cross-sensor adaptation for training, benchmarking, and deployment. Finally, we identify open challenges and future directions for VBTSs in robotics. By providing a holistic view of how hardware, AI architectures, simulation, and datasets interact, this review aims to advance tactile intelligence for contact-rich robotic tasks.

触觉感知机器人视觉传感深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。