arXiv:2512.02011cs.RO2025-12被引 8

用简化仿真训练抓取技能,再通过真实操作数据提升泛化能力。

Learning Dexterous Manipulation Skills from Imperfect Simulations

  • 先在简化模型中训练策略,自动生成正确手指动作模式。
  • 真实世界采集带触觉与本体感觉的数据,提升泛化性。
  • 可适应不同形状螺母螺丝,抗干扰能力强,适合机器人操控。

强化学习与仿真到现实的迁移在灵巧操作中取得显著进展,但复杂接触动力学和多模态信号(尤其是触觉反馈)的模拟困难仍制约其发展。本文提出一种三阶段仿真到现实框架 oots,用于多指手完成拧螺母与拧螺丝任务。第一阶段,在简化物体模型上训练强化学习策略,自发形成正确的手指步态;第二阶段,将学习到的策略作为技能基元,结合遥操作采集包含触觉与本体感知的真实数据;第三阶段,训练融合触觉信息的行为克隆策略,实现在多种几何形状螺母和螺丝上的泛化性能。跨两项任务的实验表明,该方法相比直接仿真到现实迁移具有更高的任务推进率,并在未见物体形状及外部扰动下保持稳健表现。视频与代码已公开于https://dexscrew.github.io。

原文摘要 · Abstract (English)

Reinforcement learning and sim-to-real transfer have made significant progress in dexterous manipulation. However, progress remains limited by the difficulty of simulating complex contact dynamics and multisensory signals, especially tactile feedback. In this work, we propose \ours, a sim-to-real framework that addresses these limitations and demonstrates its effectiveness on nut-bolt fastening and screwdriving with multi-fingered hands. The framework has three stages. First, we train reinforcement learning policies in simulation using simplified object models that lead to the emergence of correct finger gaits. We then use the learned policy as a skill primitive within a teleoperation system to collect real-world demonstrations that contain tactile and proprioceptive information. Finally, we train a behavior cloning policy that incorporates tactile sensing and show that it generalizes to nuts and screwdrivers with diverse geometries. Experiments across both tasks show high task progress ratios compared to direct sim-to-real transfer and robust performance even on unseen object shapes and under external perturbations. Videos and code are available on https://dexscrew.github.io.

灵巧操作仿真实现触觉反馈行为克隆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。