用关节级动态模型让机器人手在真实世界旋转复杂物体。
DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via Joint-Wise Neural Dynamics Model
- 构建关节级动态模型，用少量真实数据修复仿真与现实的差异。
- 单个仿真训练策略可适配复杂形状、高长宽比（达5.33）物体。
- 无需人工干预自动采集多样真实数据，适合真实场景部署。
实现通用的手部物体旋转仍是机器人领域的重大挑战，主要源于仿真到现实的迁移困难。灵巧操作中复杂的接触动力学造成显著的“现实差距”，以往工作受限于简单几何、有限物体尺寸和姿态、固定手腕姿势或定制机械手。本文提出一种新框架，使单一仿真训练策略可广泛应用于真实世界中的各类物体与条件。核心是关节级动力学模型，通过少量真实数据高效拟合现实差异，并相应调整仿真策略动作。该模型通过分解各关节动力学，将全局影响压缩为低维变量，基于每个关节的动态特征学习其演化，隐式捕捉整体影响，具备强数据效率与泛化能力。结合全自动数据采集策略，获取多样化真实交互数据。完整系统验证了前所未有的泛化性：单策略成功旋转复杂形状（如动物）、高长宽比（最高5.33）、小尺寸物体，且适应多变手腕姿态与旋转轴。全面真实评估及远程操控应用验证了方法的有效性与鲁棒性。
原文摘要 · Abstract (English)
Achieving generalized in-hand object rotation remains a significant challenge in robotics, largely due to the difficulty of transferring policies from simulation to the real world. The complex, contact-rich dynamics of dexterous manipulation create a "reality gap" that has limited prior work to constrained scenarios involving simple geometries, limited object sizes and aspect ratios, constrained wrist poses, or customized hands. We address this sim-to-real challenge with a novel framework that enables a single policy, trained in simulation, to generalize to a wide variety of objects and conditions in the real world. The core of our method is a joint-wise dynamics model that learns to bridge the reality gap by effectively fitting limited amount of real-world collected data and then adapting the sim policy's actions accordingly. The model is highly data-efficient and generalizable across different whole-hand interaction distributions by factorizing dynamics across joints, compressing system-wide influences into low-dimensional variables, and learning each joint's evolution from its own dynamic profile, implicitly capturing these net effects. We pair this with a fully autonomous data collection strategy that gathers diverse, real-world interaction data with minimal human intervention. Our complete pipeline demonstrates unprecedented generality: a single policy successfully rotates challenging objects with complex shapes (e.g., animals), high aspect ratios (up to 5.33), and small sizes, all while handling diverse wrist orientations and rotation axes. Comprehensive real-world evaluations and a teleoperation application for complex tasks validate the effectiveness and robustness of our approach. Website: https://meowuu7.github.io/DexNDM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。