arXiv:2603.04531cs.RO2026-03被引 4

用真实触觉数据训练机器人,让模拟中学习的抓取策略更稳定。

PTLD: Sim-to-real Privileged Tactile Latent Distillation for Dexterous Manipulation

  • 用真实触觉数据蒸馏出能处理触觉输入的状态估计器
  • 在手中旋转任务上使性能提升182%,在重定位任务中达57%提升
  • 无需模拟触觉传感器,适合缺乏真实触觉数据的复杂操作任务

触觉灵巧操作对自动化复杂家务任务至关重要,但学习有效控制策略仍具挑战。尽管近期工作依赖模仿学习,但通过机器人遥操作或动力学教学获取多指手的高质量示范成本高昂。替代方案是使用强化学习在仿真中训练技能,但快速且真实的触觉观测仿真困难。为此,我们提出PTLD:一种无需触觉仿真的模拟到现实触觉潜变量蒸馏方法。不依赖触觉传感器仿真或仅靠本体感知策略实现零样本迁移,其核心思路是利用真实世界中的特权传感器收集真实触觉策略数据,并用于蒸馏一个基于触觉输入的鲁棒状态估计器。实验表明,PTLD可显著提升仿真中训练的本体感知操作策略。在基准的手中旋转任务中,性能相比仅使用本体感知的策略提高182%;在更具挑战性的触觉手中重定向任务中,达成目标数量提升57%。

原文摘要 · Abstract (English)

Tactile dexterous manipulation is essential to automating complex household tasks, yet learning effective control policies remains a challenge. While recent work has relied on imitation learning, obtaining high quality demonstrations for multi-fingered hands via robot teleoperation or kinesthetic teaching is prohibitive. Alternatively, with reinforcement we can learn skills in simulation, but fast and realistic simulation of tactile observations is challenging. To bridge this gap, we introduce PTLD: sim-to-real Privileged Tactile Latent Distillation, a novel approach to learning tactile manipulation skills without requiring tactile simulation. Instead of simulating tactile sensors or relying purely on proprioceptive policies to transfer zero-shot sim-to-real, our key idea is to leverage privileged sensors in the real world to collect real-world tactile policy data. This data is then used to distill a robust state estimator that operates on tactile input. We demonstrate from our experiments that PTLD can be used to improve proprioceptive manipulation policies trained in simulation significantly by incorporating tactile sensing. On the benchmark in-hand rotation task, PTLD achieves a 182% improvement over a proprioception only policy. We also show that PTLD enables learning the challenging task of tactile in-hand reorientation where we see a 57% improvement in the number of goals reached over using proprioception alone. Website: https://akashsharma02.github.io/ptld-website/.

灵巧操作触觉感知模拟到现实状态估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。