用力信号引导触觉预测,让机器人更聪明地处理复杂抓握。
TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation

- 结合腕部力矩与双指触觉,预测短时触觉变化
- 在真实机器人上5个任务表现优于基线,尤其抗扰动能力强
- 轻量设计适合高频控制,适合做高动态抓取的系统
高接触交互操作要求机器人持续感知并调节随时间演化的物理交互,尤其在动态接触切换或复杂表面几何下。现有模仿学习方法虽引入触觉或力反馈提升接触感知能力,但极少建模全局力信号与局部触觉感知之间异构的时空角色。为此,我们提出TacForeSight,一种轻量级力条件触觉前瞻框架,适用于实时操作。核心是TacForceWM——一个触觉世界模型,从双指触觉观测出发,以高频腕力和力矩信号为条件,预测短时触觉潜在动态。另一关键组件是预测性触觉条件策略,利用预测潜变量作为前瞻接触先验,通过交叉注意力建模当前到未来的触觉演化,并通过触觉引导门控模块自适应融合多模态特征。通过在紧凑潜空间内进行预测,TacForeSight实现主动接触推理,具备高效实时推理能力,适用于高频操纵控制。在五个代表性任务及三种过程扰动设置下的真实机器人实验表明,该方法始终优于现有基线,尤其在动态接触干扰下表现突出。所有模型与数据集将公开于项目官网:https://tacforesight.github.io/ProjectPage。
原文摘要 · Abstract (English)
Contact-rich manipulation requires robots to continuously perceive and regulate evolving physical interactions under dynamic contact transitions or complex surface geometries. Recent imitation learning methods improve contact-aware control by incorporating tactile or force feedback, but they rarely model the asymmetric spatiotemporal roles of global force and local tactile sensing. To address this, we propose TacForeSight, a lightweight force-conditioned tactile foresight framework for real-time manipulation. The core component is TacForceWM, a tactile world model that predicts short-horizon tactile latent dynamics from dual-finger tactile observations conditioned on high-frequency wrist force and torque signals. Another key component, the Predictive Tactile-Conditioned Policy, leverages the predicted latents as anticipatory contact priors, models the current-to-future tactile evolution via cross-attention, and adaptively fuses visuo-tactile features through a tactile-guided gating module. By forecasting purely within a compact latent space, TacForeSight enables proactive contact reasoning with efficient real-time inference suitable for high-frequency manipulation control. Real-robot experiments on five representative tasks and three in-process perturbation settings show that TacForeSight consistently outperforms existing baselines, particularly under dynamic contact disturbances. All models and datasets will be made publicly available on the project website at https://tacforesight.github.io/ProjectPage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。