用残差学习高效迁移人类双手操作技能到机器人。
ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning
- 分两阶段：先通用模仿手部动作，再微调残差模块适应具体任务。
- 在复杂双臂任务上成功率、动作精度和训练效率均超现有方法。
- 构建了3300条数据的DexManipNet数据集，支持真实场景部署。
人类双手在交互中起核心作用，推动了灵巧机器人操作的研究。数据驱动的具身智能算法需要大量精确的人类级操作序列，但传统强化学习或真人遥操作难以获取。为此，我们提出ManipTrans，一种高效的两阶段方法，将人类双臂操作技能迁移到仿真中的灵巧机器人手上。该方法首先预训练一个通用轨迹模仿器以复现手部运动，随后在交互约束下微调特定残差模块，实现复杂双臂任务的高效学习与精准执行。实验表明，ManipTrans在成功率、动作保真度和效率方面均优于现有最先进方法。基于此，我们成功将多个手物交互数据集迁移至机器人手，构建了DexManipNet——一个包含3300个机器人操作片段的大规模数据集，涵盖笔帽套合、瓶盖拧开等此前未探索的任务。该数据集可扩展性强，有助于灵巧手策略进一步训练并支持真实世界应用。
原文摘要 · Abstract (English)
Human hands play a central role in interacting, motivating increasing research in dexterous robotic manipulation. Data-driven embodied AI algorithms demand precise, large-scale, human-like manipulation sequences, which are challenging to obtain with conventional reinforcement learning or real-world teleoperation. To address this, we introduce ManipTrans, a novel two-stage method for efficiently transferring human bimanual skills to dexterous robotic hands in simulation. ManipTrans first pre-trains a generalist trajectory imitator to mimic hand motion, then fine-tunes a specific residual module under interaction constraints, enabling efficient learning and accurate execution of complex bimanual tasks. Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging ManipTrans, we transfer multiple hand-object datasets to robotic hands, creating DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K episodes of robotic manipulation and is easily extensible, facilitating further policy training for dexterous hands and enabling real-world deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。