arXiv:2608.21894cs.ROcs.HC2026-08

用深度学习解析触觉数据,让机器人像人一样感知材质。

An Interpretable Deep Learning Framework for Material Perception and Classification from Multisensory Tactile Data

论文配图:An Interpretable Deep Learning Framework for Material Perception and Classification from Multisensory Tactile Data
图 1 · 摘自论文原文
  • 三阶段模型从触觉信号推导材质感知,无需人工特征提取。
  • 直接映射触觉信号到材质分类准确率接近完美。
  • 热感信号最关键,适合机器人触觉系统开发人员参考。

人类触觉感知依赖复杂的多感官线索,但触觉信号与感知表征之间的关系仍不清晰,限制了触觉在数字环境和类人机器人感知中的应用。为此,我们构建了一个由三个相互关联的深度学习模型组成的计算框架,将多模态触觉数据映射为材质感知,无需依赖手工设计特征。模型分别实现:从低层交互信号到感知属性分布(模型1),从预测属性分布到材质分类(模型2),以及直接从触觉信号到材质类别(模型3),跳过中间表征。结合集成梯度方法,该框架在保持高准确率的同时具备可解释性,揭示了驱动决策的关键感官模态。结果显示,当不设中间感知阶段时,深度学习可实现近乎完美的材质分类;但若显式建模人类感知路径,性能提升更困难。值得注意的是,热信号在所有模型中均表现出显著信息量,对材质区分具有强鲁棒性。研究为触觉信号如何生成材质感知提供了计算解释,并展示了可解释深度学习在逼近人类水平表现的同时,揭示机器人与触觉系统所需的关键感知线索。

原文摘要 · Abstract (English)

Human tactile perception relies on complex multisensory cues. Yet the relationship between tactile signals and perceptual representations remains poorly understood, limiting the integration of touch in digital environments and human-like robotic perception. To address this gap, we developed a computational framework comprising three interconnected deep learning models that map multisensory touch data to material perception, without relying on hand-crafted features. The models represent progressively different routes from tactile signals to material class: from low-level interaction signals to perceptual attribute distributions (Model 1), from predicted attribute distributions to material classification (Model 2), and directly from tactile signals to material categories, bypassing intermediate representations (Model 3). By combining deep learning with Integrated Gradients, the framework achieved high accuracy while offering interpretability, revealing which sensory modalities most strongly drive its decisions. Our results show that deep learning can approach near-perfect material classification when unconstrained by intermediate perceptual stages, but matching human-like performance is harder once those stages are modeled explicitly. Notably, thermal cues emerged as particularly informative across all models, providing robust signals for material differentiation. The results offer a computational account of how tactile signals lead to material perception and show how interpretable deep learning can both approach human-level performance and reveal cues that robotic and haptic systems need to incorporate.

触觉感知深度学习可解释性机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。