跨传感器学习统一触觉表征,实现触觉政策零样本迁移。
TactX: Learning Shared Tactile Representations Across Diverse Sensors

- 通过配对接触数据训练多模态编码器,映射不同触觉传感器到共享隐空间。
- 在4个接触任务中,跨传感器政策零样本迁移成功率达45.9%。
- 适用于多类型触觉传感器,推动无传感器依赖的触觉操作研究。
触觉传感器为高接触交互操作提供关键信息,但其表征与控制策略通常紧密耦合于特定传感器,限制了在不同机器人和硬件平台间的迁移能力。本文提出TactX框架,可在三种根本不同的传感机制(电阻式、磁式、视觉式)间学习可迁移的触觉表征。TactX通过模态专用编码器,将异构触觉观测映射至共享隐空间,利用配对接触数据提供自然的跨模态对齐信号,并联合训练所有传感器对,生成一致的隐空间。实验表明,TactX能有效对齐不同传感器的触觉表征,同时保留物体级接触信息,体现在隐空间中的传感器身份预测与物体分类准确率。在4项高接触操作任务(抓取放置、插拔、板面擦拭、物体重定向)中,使用单一传感器训练的策略可零样本迁移到物理上不同的传感器,成功率从仅视觉策略的27.5%提升至45.9%,迈向传感器无关的触觉操作。
原文摘要 · Abstract (English)
Tactile sensors provide critical information for contact-rich manipulation, yet tactile representations and policies remain tightly coupled to each specific sensor, limiting transferability across robots and hardware platforms. We propose TactX, a framework for learning a transferable tactile representation across sensors spanning three fundamentally different transduction modalities: resistive, magnetic, and vision-based. TactX maps heterogeneous tactile observations into a shared latent space through modality-specific encoders trained on paired contact data. Such paired interactions provide a natural alignment signal across modalities, and the encoders are jointly trained across all sensor pairs, inducing a consistent latent space for all sensor types. Our experiments show that TactX aligns tactile representations across sensors while preserving object-level contact information, as evidenced by sensor-identity prediction and object classification in the learned latent space. We evaluate TactX on four contact-rich manipulation tasks: pick-and-place, plug insertion, board wiping, and object reorientation, and show that policies trained with one sensor transfer zero-shot to physically distinct sensors through the shared latent. This improves the average success rate from 27.5% for vision-only policy to 45.9%, providing a step toward sensor-agnostic tactile manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。