arXiv:2606.29173cs.RO2026-06被引 1

用视觉生成触觉,解决真实触觉数据少的问题,让模型更懂物体物理属性。

TacGen: Touch Is a Necessary Dimension of Physical-World Representation -- Addressing Tactile Data Scarcity with Scalable Vision-to-Touch Alignment and Generation

论文配图:TacGen: Touch Is a Necessary Dimension of Physical-World Representation -- Addressing Tactile Data Scarcity with Scalable Vision-to-Touch Alignment and Generation
图 1 · 摘自论文原文
  • 用视觉-触觉对比对齐+残差MLP生成触觉隐变量,实现大规模触觉数据合成。
  • 在质量、密度、硬度等属性预测上,视觉+触觉比纯视觉提升显著,最大增益0.570。
  • 生成的触觉数据能大幅提升机器人操控性能,证明触觉是理解物理世界的关键。

触觉能消除视觉带来的物理属性模糊:探索性接触可恢复形状、纹理、柔软度和材质,视觉与触觉表征在腹侧视觉皮层汇聚。我们探究表示学习能否复现这一物理锚定机制。TacGen通过预设的视觉-触觉对比对齐与潜空间残差MLP生成器(从RGB生成触觉隐变量),缓解触觉数据稀缺问题。采用相同DINOv2骨干网络、划分和探测器,视觉+触觉在质量(ΔR²=+0.570)、密度(Δacc=+0.067)、硬度(+0.117)及不确定性分段力标签(ΔR²=+0.281)上均优于纯视觉,所有置信区间不包含零。相同表示将匹配容量的TACTO操控性能从0.246提升至0.979,而纯视觉容量扩展仅解释了4.5%的差距,保留95.5%。生成器跨种子表现达+0.589,真实触觉为+0.585,处于种子区间内;架构对比显示重建质量与表示效用间存在13个百分点差距。经五次种子复现、SSVTP/TVL验证、YCB-Sight迁移、三骨干检查、置换/随机特征控制、哈希验证清单及受力测量验证,证据支持触觉为接触依赖属性表征所必需的物理信息通道。

原文摘要 · Abstract (English)

Touch resolves the physical-property ambiguity left by vision: exploratory contact recovers shape, texture, compliance, and material, and visuo-haptic object representations converge in ventral visual cortex. We ask whether representation learning can reproduce this grounding. TacGen mitigates the tactile-data scarcity bottleneck by combining pre-specified V+T contrastive alignment with a latent-space residual-MLP V->T generator that synthesizes tactile latents from RGB for tactile-data scaling. With matched DINOv2 backbones, splits, and probes, V+T improves matched V-only on mass (Delta R^2=+0.570), density (Delta acc=+0.067), hardness (+0.117), and uncertainty-banded force labels (Delta R^2=+0.281); all CIs exclude zero. The same representation lifts matched-capacity TACTO manipulation 0.246->0.979 while V-only capacity scaling accounts for only 4.5% of the gap, preserving 95.5%. The generator reaches cross-seed +0.589, with real tactile +0.585 inside the seed interval; the architecture comparison shows a 13pp downstream gap between reconstruction quality and representation utility. Across five-seed SSVTP/TVL reproductions, YCB-Sight transfer, three-backbone checks, permutation/random-feature controls, hash-verified manifests, and measured-force validation checks, the evidence supports the claim that touch supplies a necessary physical evidence channel for representations of contact-dependent properties.

触觉生成多模态表示学习物理表征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。