arXiv:2606.18959cs.RO2026-06中稿 · to IROS 2026被引 1

通过物理增强的共享空间,实现触觉仿真到现实的零样本迁移。

TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer

论文配图:TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer
图 1 · 摘自论文原文
  • 构建跨模态触觉数据的共享嵌入空间,无需精确模拟原始信号。
  • 在真实传感器上实现零样本迁移,力预测误差降16.7%,形状重建误差降45.8%。
  • 适合需要高效触觉仿真与现实部署的机器人学习研究者。

触觉传感能直接测量接触交互,对机器人操作至关重要。但现有仿真器难以准确建模触觉传感器复杂的形变与信号转换机制,严重阻碍了机器人学习中的仿真到现实迁移。为此,我们提出一种多模态表征学习框架,在共享潜在空间中对齐异构触觉模态,无需精确模拟原始信号即可保留关键接触信息。该方法使用模态专用编码器将模拟的穿刺深度与真实的电容信号等多样触觉观测映射至统一嵌入空间,通过自重构与交叉重构目标及对比对齐进行训练,生成模态不变且信息丰富的表示。我们在压头形状识别、力值预测和几何重建任务上评估所学嵌入,仅在仿真中训练,直接在真实传感器数据上测试。结果表明,可在物理差异显著的表示间实现零样本仿真到现实迁移。引入多物理仿真模态可生成更丰富嵌入,使力预测误差降低16.7%,形状重建误差降低45.8%。最后,我们发布基于Warp的惩罚式触觉仿真模型,集成于Isaac Lab,支持可扩展的触觉数据生成。

原文摘要 · Abstract (English)

Tactile sensing provides direct measurements of contact interactions that are essential for robotic manipulation. However, current simulators lack the fidelity to faithfully model the complex deformation and transduction mechanics of tactile sensors, severely hindering sim-to-real transfer in robot learning pipelines. To address this challenge, we propose a multi-modal representation learning framework that aligns heterogeneous tactile modalities within a shared latent space, eliminating the need for accurate raw-signal simulation while preserving relevant contact information. Our approach employs modality-specific encoders to project diverse tactile observations, such as simulated penetration depth and real-world capacitance, into a common embedding space. The model is trained using self- and cross-reconstruction objectives alongside contrastive alignment, encouraging modality-invariant yet information-rich representations. We evaluate the learned embeddings on indenter shape identification, force prediction, and geometric reconstruction tasks, training exclusively in simulation and testing directly on real sensor measurements. Our results demonstrate zero-shot sim-to-real transfer across physically dissimilar representations. Furthermore, incorporating multi-physics simulation modalities yields more informative embeddings that transfer across diverse downstream tasks, demonstrating a 16.7% reduction in force prediction error and a 45.8% reduction in shape reconstruction error. Finally, we release an efficient Warp-based implementation of a penalty-based tactile simulation model for Isaac Lab, enabling scalable tactile data generation.

触觉感知仿真迁移多模态学习机器人学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。