让机器人通过视觉感知触觉，实现共情式交互
Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance
- 构建视觉与触觉表征的多层级对齐模型
- 从图像精准预测机器人手上1140个触点的毫米级触感
- 可推广至人类手部观察，支持反射式触觉响应
观察他人身体受触可引发观察者相应的触觉感受，这一现象称为镜像触觉，有助于共情与社会认知。该跨模态共振被认为依赖于视觉与躯体感觉皮层之间的结构对应，但现有机器人系统缺乏计算框架来实现此原理。本文提出镜像触觉网络（Mirror Touch Net），通过语义、分布和几何多层次约束，对齐视觉与触觉表征，实现了从RGB图像对机器人手部1,140个触点（taxels）的毫米级触觉信号预测。流形分析表明，这些约束使视觉表征的几何结构更接近触觉流形，降低跨模态映射复杂度。将该对齐框架扩展至对人类手部的跨域观测后，可实现触觉预测与对观察到的人类触碰的反射响应。研究将神经层面的视觉-触觉共振原理与机器人感知相结合，为预期触觉与共情式人机交互提供了可解释路径。代码已开源。
原文摘要 · Abstract (English)
Observing touch on another's body can elicit corresponding tactile sensations in the observer, a phenomenon termed mirror touch that supports empathy and social perception. This visuo-tactile resonance is thought to rely on structural correspondence between visual and somatosensory cortices, yet robotic systems lack computational frameworks that instantiate this principle. Here we demonstrate that cortical correspondence can be operationalized to endow robots with mirror touch. We introduce Mirror Touch Net, which imposes semantic, distributional and geometric alignment between visual and tactile representations through multi-level constraints, enabling prediction of millimetre-scale tactile signals across 1,140 taxels on a robotic hand from RGB images. Manifold analysis reveals that these constraints reshape visual representations into geometry consistent with the tactile manifold, reducing the complexity of cross-modal mapping. Extending this alignment framework to cross-domain observations of human hands enables tactile prediction and reflexive responses to observed human touch. Our results link a neural principle of visuo-tactile resonance to robotic perception, providing an explainable route towards anticipatory touch and empathic human-robot interaction. Code is available at https://github.com/fun0515/Mirror-Touch-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。