arXiv:2609.05266cs.RO2026-09

让机器人通过触觉反馈实时修正动作,提升复杂操作成功率。

TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulation

论文配图:TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulation
图 1 · 摘自论文原文
  • 用触觉预测指导动作实时修正,而非仅预测未来触觉。
  • 在5个真实任务中平均成功率从22%提升至64%。
  • 修正成本仅为重生成动作的1/20.7,适合高精度操作场景。

世界-动作模型通过预测未来观测来指导动作生成,但以视觉为中心的预测会遗漏决定接触密集型操作的关键局部接触信息。然而,简单地将未来触觉观测作为额外视图进行预测,仅恢复了可实现增益的三分之一。这一差距源于时间错配:预测早于执行,而触觉反馈在执行期间到达。我们提出TacPAC,将触觉预测转化为实时动作修正。当基础模型规划好一个动作片段后,TacPAC缓存该计划所依赖的预测接触及其自身表示,随后触觉专家将每个新观测的触觉图像与该缓存对比,对尚未执行的动作进行修正。反馈被解释为与计划预期相比,而非孤立处理;一次修正只需遍历一次缓存,比重新生成动作片段快20.7倍。在五个真实机器人任务(包括精密插入、易碎品处理、物体翻转和长时程操作)中,TacPAC在所有任务上均表现更优,使平均成功率从基础视觉模型的22%提升至64%。代码已公开于https://github.com/LogosRoboticsGroup/TacPAC。

原文摘要 · Abstract (English)

World-action models guide action generation with predicted future observations, but vision-centric predictions miss the local contact cues that decide contact-rich manipulation. However, naively predicting future tactile observations as additional views recovers only a third of the achievable gain in our experiments. This gap reflects a timing mismatch: predictions precede execution, while tactile feedback arrives during it. We introduce TacPAC, which turns tactile prediction into real-time action correction. Once the base model has planned an action chunk, TacPAC caches the predicted contact that plan was conditioned on together with the plan's own representation, and a tactile expert reads each newly observed tactile image against that cache to correct the actions not yet executed. Feedback is thus interpreted against what the plan anticipated rather than in isolation, and one correction is a single pass over that cache, $20.7\times$ cheaper than regenerating the chunk. On five real-robot tasks spanning precision insertion, fragile-object handling, object reorientation, and long-horizon manipulation, TacPAC leads every task and raises the average from 22% for its vision-only base model to 64%. Code is available at https://github.com/LogosRoboticsGroup/TacPAC.

触觉感知动作修正机器人操作世界模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。