无需配对数据，用序列匹配实现跨机器人模仿
One-Shot Imitation under Mismatched Execution
- 用序列级最优传输自动对齐人类与机器人的动作轨迹
- 在模拟和真实机械手上实现任务成功率提升超50%
- 适合做跨设备机器人模仿的科研与工程人员
人类示范作为提示是编程机器人完成长时序操作任务的强大方式。然而，由于运动风格和物理能力差异导致的执行不匹配，将示范转化为可执行动作面临巨大挑战。现有方法或依赖难以扩展的成对数据，或严重依赖帧级视觉相似性，实际中常失效。为此，我们提出RHyME框架，通过序列级最优传输成本函数自动匹配人类与机器人轨迹。给定长时序机器人示范，RHyME通过检索和组合短时序人类片段，合成语义等价的人类视频，从而在无需配对数据的情况下有效训练策略。RHyME在模拟环境及真实人手实验中成功模仿多种跨体机器人示范，任务成功率较之前方法提升超过50%。代码与数据集已开源：https://portal-cornell.github.io/rhyme/
原文摘要 · Abstract (English)
Human demonstrations as prompts are a powerful way to program robots to do long-horizon manipulation tasks. However, translating these demonstrations into robot-executable actions presents significant challenges due to execution mismatches in movement styles and physical capabilities. Existing methods for human-robot translation either depend on paired data, which is infeasible to scale, or rely heavily on frame-level visual similarities that often break down in practice. To address these challenges, we propose RHyME, a novel framework that automatically pairs human and robot trajectories using sequence-level optimal transport cost functions. Given long-horizon robot demonstrations, RHyME synthesizes semantically equivalent human videos by retrieving and composing short-horizon human clips. This approach facilitates effective policy training without the need for paired data. RHyME successfully imitates a range of cross-embodiment demonstrators, both in simulation and with a real human hand, achieving over 50% increase in task success compared to previous methods. We release our code and datasets at https://portal-cornell.github.io/rhyme/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。