将人类手势迁移到机器人手，实现高泛化能力的灵巧操作。
DexH2R: Task-oriented Dexterous Manipulation from Human to Robots
- 通过人体手势迁移+任务导向残差策略，减少对复杂采集设备依赖。
- 在模拟与真实场景中均超越现有方法40%性能表现。
- 支持测试时引导新任务，适合需快速适应新场景的机器人系统。
灵巧操作是人类核心能力之一，可与多种物体交互。近年来，基于人类示范和遥操作的学习方法推动了机器人在该领域的进展，但这些方法要么需要高成本的人类数据采集（如眼-机器人接触），要么在面对新场景时泛化能力差。为此，我们提出DexH2R框架，结合人体手部动作迁移与任务导向残差动作策略，通过弥合人机灵巧手之间的形态差距来提升任务表现。具体而言，DexH2R直接从迁移后的基础动作和任务导向奖励中学习残差策略，无需耗时的遥操作系统。此外，通过输入人体手与物体的目标轨迹，可在测试时提供引导，使机器人手以高泛化性掌握新技能。大量仿真与真实世界实验表明，本方法在多种设置下均优于先前最先进方法40%。
原文摘要 · Abstract (English)
Dexterous manipulation is a critical aspect of human capability, enabling interaction with a wide variety of objects. Recent advancements in learning from human demonstrations and teleoperation have enabled progress for robots in such ability. However, these approaches either require complex data collection such as costly human effort for eye-robot contact, or suffer from poor generalization when faced with novel scenarios. To solve both challenges, we propose a framework, DexH2R, that combines human hand motion retargeting with a task-oriented residual action policy, improving task performance by bridging the embodiment gap between human and robotic dexterous hands. Specifically, DexH2R learns the residual policy directly from retargeted primitive actions and task-oriented rewards, eliminating the need for labor-intensive teleoperation systems. Moreover, we incorporate test-time guidance for novel scenarios by taking in desired trajectories of human hands and objects, allowing the dexterous hand to acquire new skills with high generalizability. Extensive experiments in both simulation and real-world environments demonstrate the effectiveness of our work, outperforming prior state-of-the-arts by 40% across various settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。