无需建模剪切动力学,用仿真生成真实触觉剪切数据
SimShear: Sim-to-Real Shear-based Tactile Servoing
- 用条件GAN将无剪切的仿真触觉图转为含剪切的真实图像
- 在两种任务中保持接触误差1-2毫米,剪切感知关键时仍有效
- 适合做低成本机器人触觉控制,尤其需动态交互的场景
我们提出SimShear,一种用于触觉控制的仿真到现实迁移方法,可在不显式建模剪切动力学的情况下利用剪切信息。剪切源于接触表面的横向运动,对涉及动态物体交互的任务至关重要,但模拟困难。为此,我们引入shPix2pix——一种剪切条件的U-Net GAN,将不含剪切的仿真触觉图像与剪切信息向量联合转换为包含剪切变形的真实图像。该方法在仿真触觉图像生成及位姿/剪切预测上优于基线pix2pix。我们在一对低成本桌面机械臂(配备基于视觉的触觉传感器)上应用SimShear,完成两项任务:(i) 触觉跟踪任务,跟随臂追踪领头臂移动的表面;(ii) 协同共举任务,双臂共同抓持物体,领头臂沿预设轨迹运动。在多种轨迹下,接触误差维持在1至2毫米之间,验证了在刚体仿真器中实现仿真到现实剪切建模的可行性,为触觉机器人仿真开辟新方向。
原文摘要 · Abstract (English)
We present SimShear, a sim-to-real pipeline for tactile control that enables the use of shear information without explicitly modeling shear dynamics in simulation. Shear, arising from lateral movements across contact surfaces, is critical for tasks involving dynamic object interactions but remains challenging to simulate. To address this, we introduce shPix2pix, a shear-conditioned U-Net GAN that transforms simulated tactile images absent of shear, together with a vector encoding shear information, into realistic equivalents with shear deformations. This method outperforms baseline pix2pix approaches in simulating tactile images and in pose/shear prediction. We apply SimShear to two control tasks using a pair of low-cost desktop robotic arms equipped with a vision-based tactile sensor: (i) a tactile tracking task, where a follower arm tracks a surface moved by a leader arm, and (ii) a collaborative co-lifting task, where both arms jointly hold an object while the leader follows a prescribed trajectory. Our method maintains contact errors within 1 to 2 mm across varied trajectories where shear sensing is essential, validating the feasibility of sim-to-real shear modeling with rigid-body simulators and opening new directions for simulation in tactile robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。