arXiv:2607.01684cs.RO2026-07

让机器人通过想象触觉来提升操作能力,无需真实触觉传感器。

Imagining the Sense of Touch: Touch-Informed Manipulation via Imagined Tactile Representations

论文配图:Imagining the Sense of Touch: Touch-Informed Manipulation via Imagined Tactile Representations
图 1 · 摘自论文原文
  • 用视觉和本体感知预测触觉信号,生成虚拟触觉指导操作。
  • 实测中触觉想象使敏感任务成功率平均提升44.4%(力场)和23.3%(纹理)。
  • 适合缺乏触觉硬件但需精细接触控制的机器人场景。

触觉感知能显著提升接触密集型机器人操作性能,但其实际部署受限于触觉硬件的脆弱性、校准需求与维护负担。这引出一个根本问题:机器人能否在不使用触觉传感器的情况下受益于触觉知识?我们提出TacImag框架,通过视觉与本体感知预测触觉观测,并利用生成信号指导操作策略。该框架基于成对的视觉-触觉演示进行训练,测试时仅需视觉输入即可实现触觉感知引导的操作。我们在六项模拟任务和四项真实世界操作任务上评估了TacImag。在仿真与真实实验中,想象的触觉信号持续提升操作性能,且无需真实触觉硬件。真实实验中,想象的力场使接触敏感任务成功率平均提升44.4%,想象的触觉图像使纹理敏感任务提升23.3%,表明触觉想象的效果强依赖于触觉表征与任务需求的关系。结果进一步显示,触觉想象并非简单恢复缺失的触觉测量,而是作为一种接触感知监督机制,将细微的视觉交互线索转化为操作策略更易利用的表示。

原文摘要 · Abstract (English)

Tactile sensing can substantially improve contact-rich robotic manipulation, yet its practical deployment remains limited by the fragility, calibration requirements, and maintenance burden of tactile hardware. This raises a fundamental question: can robots benefit from tactile knowledge without requiring tactile sensors at deployment? We present TacImag, a tactile imagination framework that predicts tactile observations from vision and proprioception and uses the generated signals to guide manipulation policies. Trained from paired visuotactile demonstrations, TacImag enables touch-informed manipulation using only visual observations at test time. We evaluate TacImag on six simulated and four real-world manipulation tasks. Across simulation and real-world experiments, imagined tactile observations consistently improve manipulation performance without requiring tactile hardware. In real-world experiments, imagined force fields improve contact-sensitive tasks by 44.4% on average, whereas imagined tactile images improve texture-sensitive tasks by 23.3%, revealing that the effectiveness of tactile imagination depends strongly on the relationship between tactile representation and task requirements. Our results further suggest that tactile imagination does not simply recover missing tactile measurements. Instead, it acts as a form of contact-aware supervision that transforms subtle visual interaction cues into representations that are easier for manipulation policies to exploit.

触觉想象机器人操作视觉引导无传感器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。