用触觉反馈提升视觉机器人策略，实现在复杂操作中的高成功率。
OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies
- 通过残差修正方式，将触觉信号融入预训练视觉策略，无需修改原策略。
- 在4个真实接触任务中，成功率从5%-40%提升至85%-100%，仅需40-80分钟。
- 支持多种机器人、传感器和任务，实现跨域泛化，适合实际部署场景。
从人类视频、遥操作和机器人演示中学习的视觉策略可提供可扩展的动作先验，但在依赖接触的操控任务中表现不佳，因成功高度依赖局部力和接触几何。触觉感知能提供互补信息，但触觉数据采集成本高，且难以在不同传感器、机器人和任务间泛化。我们提出OmniTacTune，一种无需依赖策略的现实世界强化学习流程，通过残差校正将触觉反馈适配到预训练的视觉策略。该方法采用两阶段设计：首先基于自主基策略的滚动回放启动触觉感知学习，再通过在线交互学习轻量级触觉残差策略。大量实验表明，OmniTacTune在多样接触任务、视觉基策略及触觉表示间具有良好泛化能力。在四个真实世界接触任务中，将视觉基策略成功率从5%-40%提升至85%-100%，仅耗时40-80分钟，为高效适配触觉反馈提供了可行路径。项目页：https://colinyu1.github.io/omnitactune-site/
原文摘要 · Abstract (English)
Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first bootstraps tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5-40% success to 85-100% within 40-80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune-site/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。