用触觉反馈实时引导机器人精细操作,提升抓取与接触任务成功率。
TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance
- 在低维动作空间融合视觉与触觉信号,分两阶段优化动作生成。
- 在5个高接触任务中显著超越现有方法,成功率达87%以上。
- 适合需要精准触觉交互的机器人操作场景,如穿鞋带、芯片传递。
精细且高接触密度的操作对机器人仍具挑战,主要因触觉反馈利用不足。为此,我们提出TouchGuide,一种新的跨策略视觉-触觉融合范式,将多模态信息融合于低维动作空间。TouchGuide在推理时分两阶段工作:首先仅用视觉输入生成粗略、视觉合理的动作;其次,通过特定任务的接触物理模型(CPM)提供触觉引导,修正并优化动作以满足真实物理接触条件。CPM基于少量专家示范通过对比学习训练,输出触觉感知的可行性分数,引导采样过程向满足物理约束的动作收敛。为高效获取高质量低成本数据,我们引入TacUMI数据采集系统,利用刚性指尖直接获取触觉反馈,实现可靠数据收集。在鞋带穿系、芯片交接等五个高接触任务上的大量实验表明,TouchGuide持续且显著优于当前最优视觉-触觉策略。
原文摘要 · Abstract (English)
Fine-grained and contact-rich manipulation remain challenging for robots, largely due to the underutilization of tactile feedback. To address this, we introduce TouchGuide, a novel cross-policy visuo-tactile fusion paradigm that fuses modalities within a low-dimensional action space. Specifically, TouchGuide operates in two stages to guide a pre-trained diffusion or flow-matching visuomotor policy at inference time. First, the policy produces a coarse, visually-plausible action using only visual inputs during early sampling. Second, a task-specific Contact Physical Model (CPM) provides tactile guidance to steer and refine the action, ensuring it aligns with realistic physical contact conditions. Trained through contrastive learning on limited expert demonstrations, the CPM provides a tactile-informed feasibility score to steer the sampling process toward refined actions that satisfy physical contact constraints. Furthermore, to facilitate TouchGuide training with high-quality and cost-effective data, we introduce TacUMI, a data collection system. TacUMI achieves a favorable trade-off between precision and affordability; by leveraging rigid fingertips, it obtains direct tactile feedback, thereby enabling the collection of reliable tactile data. Extensive experiments on five challenging contact-rich tasks, such as shoe lacing and chip handover, show that TouchGuide consistently and significantly outperforms state-of-the-art visuo-tactile policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。