让机器人学会动态抓取与人机交接，模拟真实交互动作
Modeling Dynamic Hand-Object Interactions with Applications to Human-Robot Handovers
- 用物理仿真+强化学习实现手物动态抓握与移动
- 合成数据使训练多样性提升100倍，性能媲美真实数据
- 适合人机协作、虚拟现实等需要自然交互的场景
人类频繁抓握、操作并移动物体。交互系统可辅助完成此类任务，应用于具身智能、人机交互和虚拟现实。然而，当前手物生成方法多忽略动态特性，仅关注静态抓握。本文第一部分提出动态抓握生成，使手部将物体抓起并移动至目标姿态。采用物理仿真与强化学习解决该问题，并扩展至双手操作及可动物体，要求双手精细协同。第二部分研究人机交接，将捕捉的人体运动融入仿真，引入学生-教师框架以适应人类行为，并实现从仿真到现实的迁移。为克服数据稀缺，生成合成交互数据，使训练多样性提升100倍。用户研究表明，基于合成数据与真实数据训练的策略无显著差异。
原文摘要 · Abstract (English)
Humans frequently grasp, manipulate, and move objects. Interactive systems assist humans in these tasks, enabling applications in Embodied AI, human-robot interaction, and virtual reality. However, current methods in hand-object synthesis often neglect dynamics and focus on generating static grasps. The first part of this dissertation introduces dynamic grasp synthesis, where a hand grasps and moves an object to a target pose. We approach this task using physical simulation and reinforcement learning. We then extend this to bimanual manipulation and articulated objects, requiring fine-grained coordination between hands. In the second part of this dissertation, we study human-to-robot handovers. We integrate captured human motion into simulation and introduce a student-teacher framework that adapts to human behavior and transfers from sim to real. To overcome data scarcity, we generate synthetic interactions, increasing training diversity by 100x. Our user study finds no difference between policies trained on synthetic vs. real motions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。