首个跨传感器通用触觉策略,让机器人触觉技能可迁移。
FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation

- 用异构编码器统一不同触觉信号,通过共享触觉Transformer建模。
- 预训练3000小时数据后,在已见传感器上提升17.2%,未见传感器提升31%。
- 适合做触觉机器人操控的科研与工程人员参考。
尽管基于视觉的通用机器人策略已取得成功,现有触觉策略仍受限于固定硬件和传感器配置。这是因为触觉信号在不同设备间差异巨大,导致跨传感器泛化困难。本文提出FTP-1,首个预训练的通用触觉基础策略,可跨多种传感器与机械结构迁移触觉操作能力。FTP-1支持图像、阵列和状态三种触觉输入,通过异构编码器将它们映射为统一的形态感知隐向量,由共享的触觉Transformer专家联合建模。在涵盖21种传感器、26个数据源的约3000小时触觉操作数据上预训练,该模型学习到可超越预训练传感器的触觉技能。在5种下游硬件配置的微调实验中,对已见传感器设置提升17.2%成功率,意外地在两种未见过的触觉传感器上实现31%的性能增益。FTP-1建立了首个统一的触觉操作基础基准,为未来触觉策略提供模型级起点。预训练模型、数据集、代码及可视化详见 https://ftp1-policy.github.io。
原文摘要 · Abstract (English)
Despite the success of vision-based generalist robotic policies, existing tactile-based policies remain tied to fixed embodiments and sensor setups. This is because tactile signals are highly heterogeneous across hardware, making cross-sensor generalization difficult. We present FTP-1,the first generalist foundation tactile policy pretrained to acquire transferable tactile manipulation abilities across diverse sensors and embodiments. FTP-1 supports varied tactile inputs, including image-, array-, and state-based signals, by using heterogeneous encoders to project them into unified morphology-aware latent tokens that are jointly modeled by a shared tactile Transformer expert. Pretrained on around 3,000 hours of tactile manipulation data aggregated from 26 data sources, spanning human and robot demonstrations across 21 sensors, FTP-1 learns tactile skills that transfer beyond the sensors seen during pretraining. Across downstream finetuning experiments spanning 5 hardware configurations, FTP-1 improves contact-rich manipulation on seen sensor setups by +17.2% and, surprisingly, transfers to two previously unseen tactile-sensor setups, achieving a +31% gain in success rate. FTP-1 establishes the first unified foundation baseline for tactile manipulation, providing future tactile policies with a shared model-level starting point. Pretrained models, datasets, training code and more visualization at https://ftp1-policy.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。