让机器人通过触觉信号更精准预测操作动作，避免视觉误导。
Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
- 用不对称注意力机制分离触觉与视觉信息，防止触觉干扰视觉建模
- 在真实机器人任务中成功率提升至49.2%，在模拟任务中达32.7%
- 适用于需要精细触觉反馈的抓取、对齐等复杂操作场景
世界动作模型（WAMs）联合预测未来视觉观测和动作，但仅靠视觉常遗漏滑移、卡滞、接触方向变化及细微错位等物理状态。触觉信号可揭示这些隐藏状态，但直接注入触觉信息会因数据量小而破坏视觉动态建模，这种现象称为触觉污染。本文提出触觉感知的世界动作模型Tactile-WAM，采用不对称注意力机制，阻止视频查询访问触觉键，同时保留动作查询对触觉的访问。通过接触变化感知偏差强化动作注意力。由于触觉像素变化不可靠反映接触变化，我们以观测代理变化驱动注意力偏差，未来代理监督则保持预测触觉表示中的动作相关接触动态。在ManiFeel数据集上，视觉路径隔离使步匹配20K检查点处的均方误差相比纯视觉轨迹降低21.8%，且地面真实视频质量无显著变化。完整模型将平均成功率从15.6%提升至32.7%，其中VideoClean带来最大提升。在五个真实机器人任务中，Tactile-WAM实现49.2%的成功率。
原文摘要 · Abstract (English)
World Action Models (WAMs) jointly predict future visual observations and actions, but visual futures alone often miss slip, jamming, contact-direction changes, and subtle misalign- ment in contact-rich manipulation. Tactile signals reveal these hidden physical states, yet naive tactile-token injection can disrupt visual dynamics modeling due to the limited scale of tactile data, a phenomenon we term tactile pollution. We in- troduce Tactile-WAM, which uses asymmetric attention to block video queries from tactile keys while preserving tac- tile access for action queries. A contact-change-aware bias further strengthens action attention to touch. Because tactile pixel changes do not reliably reflect contact changes, we derive Observed proxy changes drive the attention bias, while future- proxy supervision preserves action-relevant contact dynamics in predicted tactile representations. On ManiFeel, visual-path isolation reduces deviation from the RGB-only trajectory by 21.8% in MSE at the step-matched 20K checkpoint without a statistically detectable change in ground-truth video qual- ity. The full model improves average success from 15.6% to 32.7%, with VideoClean providing the largest gain. On five real-robot tasks, Tactile-WAM achieves 49.2% success.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。