从人类手部动作学习机器人灵巧操作，有效解决人机动作差异问题。
Learning to Transfer Human Hand Skills for Robot Manipulations
- 构建人手、机器人手与物体的联合运动流形，实现三者间动作映射。
- 通过合成伪监督三元组，在真实机器人上显著提升操作成功率。
- 适合研究机器人灵巧操作与人机动作迁移的开发者参考。
我们提出一种从人类手部动作示范中教授机器人灵巧操作的方法。与仅依赖运动学信息而忽略机器人与物体交互合理性的现有方法不同，本方法直接从人类动作示范中推断出合理的机器人操作动作。为解决人手与机器人系统间的形态差异（embodiment gap），我们的方法学习一个联合运动流形，将人类手部运动、机器人手部动作和物体运动在三维空间中统一建模，从而实现任意一者的动作推断。核心思想是生成合成的伪监督三元组，即人为配对的人类、物体和机器人运动轨迹。通过在真实机器人手上进行的实验，验证了该数据驱动的重定向方法显著优于传统技术，有效弥合了人手与机械手之间的动作鸿沟。
原文摘要 · Abstract (English)
We present a method for teaching dexterous manipulation tasks to robots from human hand motion demonstrations. Unlike existing approaches that solely rely on kinematics information without taking into account the plausibility of robot and object interaction, our method directly infers plausible robot manipulation actions from human motion demonstrations. To address the embodiment gap between the human hand and the robot system, our approach learns a joint motion manifold that maps human hand movements, robot hand actions, and object movements in 3D, enabling us to infer one motion component from others. Our key idea is the generation of pseudo-supervision triplets, which pair human, object, and robot motion trajectories synthetically. Through real-world experiments with robot hand manipulation, we demonstrate that our data-driven retargeting method significantly outperforms conventional retargeting techniques, effectively bridging the embodiment gap between human and robotic hands. Website at https://rureadyo.github.io/MocapRobot/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。