arXiv:2411.04005cs.RO2024-11CoRL被引 76

用人体手势数据训练机器人抓取,实现精准灵巧操作。

Object-Centric Dexterous Manipulation from Human Motion Data

  • 分层策略:先生成类人手腕动作,再训练机器人手指控制。
  • 10种日常物品测试,新物体和目标状态均表现良好。
  • 从仿真到真实双臂机器人成功部署,适合实际应用。

灵巧操作物体以达成目标状态是重要基础技能。人类手部动作展现出高超的操纵能力,为训练多指机器人提供了宝贵数据。然而,由于人手与机器人手之间的具身差距,仍存在显著挑战。本文提出一种分层策略学习框架,利用大规模人体手势捕捉数据训练面向物体的灵巧机器人操作。核心是一个高层轨迹生成模型,基于大量人体手势数据,根据期望的物体目标状态生成类人手腕运动。在生成的手腕运动引导下,通过深度强化学习训练底层手指控制器,使其基于机器人本体特性与物体物理交互以达成目标。在10种家用物品上广泛评估,该方法不仅表现优异,还展现出对新物体几何形状和目标状态的泛化能力。此外,所学策略成功从仿真迁移到真实世界双臂灵巧机器人系统,验证了其在真实场景中的适用性。

原文摘要 · Abstract (English)

Manipulating objects to achieve desired goal states is a basic but important skill for dexterous manipulation. Human hand motions demonstrate proficient manipulation capability, providing valuable data for training robots with multi-finger hands. Despite this potential, substantial challenges arise due to the embodiment gap between human and robot hands. In this work, we introduce a hierarchical policy learning framework that uses human hand motion data for training object-centric dexterous robot manipulation. At the core of our method is a high-level trajectory generative model, learned with a large-scale human hand motion capture dataset, to synthesize human-like wrist motions conditioned on the desired object goal states. Guided by the generated wrist motions, deep reinforcement learning is further used to train a low-level finger controller that is grounded in the robot's embodiment to physically interact with the object to achieve the goal. Through extensive evaluation across 10 household objects, our approach not only demonstrates superior performance but also showcases generalization capability to novel object geometries and goal states. Furthermore, we transfer the learned policies from simulation to a real-world bimanual dexterous robot system, further demonstrating its applicability in real-world scenarios. Project website: https://cypypccpy.github.io/obj-dex.github.io/.

灵巧操作人体数据分层控制仿真到现实

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。