用物理精确模拟光学触觉传感器,实现快速真实世界校准。
DOT-Sim: Differentiable Optical Tactile Simulation with Precise Real-to-Sim Physical Calibration

- 基于材料点法建模软体传感器的弹性行为
- 支持大变形非线性响应,零样本部署准确率达85%以上
- 适合机器人触觉感知与仿真训练,尤其擅长复杂接触场景
光学触觉传感器因高柔性和复杂的光学特性,仿真极具挑战。本文提出可微分光学触觉仿真框架DOT-Sim,采用物质点法(MPM)将传感器建模为弹性材料,精准捕捉其物理行为。该方法仅需少量演示即可在数分钟内完成物理参数校准,远快于现有方法。相比基线,支持更大且非线性的形变。针对光学响应,提出学习真实空闲状态的残差图像来模拟光学信号。通过一系列零样本仿真到现实任务验证,实验表明:DOT-Sim能准确复现DenseTact传感器的物理动态;在密集接触场景中生成逼真的光学输出;直接部署仿真训练的分类器,在挑战性物体上实现85%分类准确率,嵌入式肿瘤检测达90%;基于仿真演示训练的策略可实现轨迹跟踪,平均误差低于0.9毫米。
原文摘要 · Abstract (English)
Simulating optical tactile sensors presents significant challenges due to their high deformability and intricate optical properties. To address these issues and enable a physically accurate simulation, we propose DOT-Sim: Differentiable Optical Tactile Simulation. Unlike prior simulators that rely on simplified models of deformable sensors, DOT-Sim accurately captures the physical behavior of soft sensors by modeling them as elastic materials using the Material Point Method (MPM). DOT-Sim enables rapid calibration of optical tactile sensor simulation using a small number of demonstrations within minutes, which is substantially faster than existing methods. Compared to current baselines, our approach supports much larger and non-linear deformations. To handle the optical aspect, we propose a novel approach to simulating optical responses by learning a residual image relative to the real-world idle state. We validate the physical and visual realism of our method through a series of zero-shot sim-to-real tasks. Our experiments show that DOT-Sim (1) accurately replicates the physical dynamics of a DenseTact optical tactile sensor in reality, (2) generates realistic optical outputs in contact-rich scenarios, (3) enables direct deployment of simulation-trained classifiers in the real world, achieving 85% classification accuracy on challenging objects and 90% accuracy in embedded tumor-type detection, and (4) allows precise trajectory following with a policy trained from demonstrations in simulation, with an average error of less than 0.9 mm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。