无需模拟触觉传感器,通过隐空间对齐实现机器人触觉零样本迁移。
Tactile Sim2Real without Tactile Simulation via Bottlenecked Latent Reconstruction

- 用虚拟触觉原语替代真实传感器建模,训练时无需具体传感器信息。
- 在三个接触任务中性能接近原语直接模拟,硬件测试成功率85%-97.5%。
- 适用于无传感器建模经验的研究者,尤其适合快速部署触觉系统。
机器人传感器设计多样且更新迅速,为每种传感器构建仿真模型需大量领域知识,且近似计算会降低信号保真度。我们提出基于瓶颈隐空间重建的Sim2Real框架(SBLR),完全避免针对特定传感器的仿真:(1) 在易构建的仿真原语传感器(如点云与指尖力)上训练策略;(2) 推理时通过未配对的仿真与真实世界随机交互数据,利用基于修正流的变换网络学习真实传感器隐空间与原语隐空间的对齐。策略训练分两阶段:先从原语隐空间学习,再通过瓶颈重建适应真实传感器的信息损失。三组接触密集任务的仿真实验表明,SBLR性能达到或逼近拥有直接触觉仿真访问权限的原语水平。硬件实验在GelSight Mini和DIGIT传感器上完成插销与齿轮啮合任务,零样本成功率达85%-97.5%,无需任何传感器特异性建模或校准,优于基于物理的触觉仿真基线7.5%-15%。
原文摘要 · Abstract (English)
Robot sensor designs, particularly tactile sensors, are highly diverse and evolve rapidly. Modeling each sensor in simulation demands substantial domain expertise and computational approximations can degrade the fidelity of the simulated signals. We propose Sim2Real via Bottlenecked Latent Reconstruction (SBLR), a framework that avoids sensor-specific simulation entirely by (1) training policies on a simulator-native oracle sensor that is easy to construct without modeling any particular sensor (e.g. we use a point-cloud and finger-tip forces as a tactile oracle), and (2) aligning real sensor latent embeddings to those of the oracle sensor at inference time. Policy training proceeds in two-stage: the policy first learns from the oracle sensor latents, then a bottlenecked latent reconstruction adapts it to the information loss expected when using the real sensor instead of the oracle. The alignment between oracle and real sensor is learned from unpaired random-play data collected in both simulation and the real world, using rectified-flow-based transformation networks trained on nearest-neighbor pseudo-pairs. Simulation experiments on three contact-rich tasks show that SBLR matches or approaches the performance of an oracle with direct access to tactile simulation. Hardware experiments on Peg Insertion and Gear Meshing with GelSight Mini and DIGIT sensors demonstrate 85-97.5% zero-shot success without requiring any sensor-specific modeling or calibration, outperforming a physics-based tactile simulation baseline by 7.5-15%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。