纯合成数据训练通用手物追踪控制器，无需真人演示。
Learning Generalizable Hand-Object Tracking from Synthetic Demonstrations
- 用合成数据生成多样化手物轨迹，实现无真人示范的训练。
- 在真实机器人上成功追踪长时序复杂操作，如物体重排和翻转。
- 适合研究通用抓取控制与合成数据训练的学者使用。
我们提出一种仅依赖合成数据学习通用手物追踪控制器的系统，无需任何人类示范。方法包含两项关键贡献：(1) HOP 手物规划器，可生成多样化的手物运动轨迹；(2) HOT 手物追踪器，通过强化学习与交互模仿学习实现从合成数据到物理世界的迁移，输出基于目标手物状态的通用控制器。该方法适用于多种物体形状与手部形态。大量实验表明，该方法使灵巧手能有效追踪具有挑战性的长时序任务，包括物体重排与敏捷的手中翻转。这些成果标志着向完全由合成数据驱动的可扩展基础操控控制器迈出重要一步，突破了长期制约灵巧操作发展的数据瓶颈。
原文摘要 · Abstract (English)
We present a system for learning generalizable hand-object tracking controllers purely from synthetic data, without requiring any human demonstrations. Our approach makes two key contributions: (1) HOP, a Hand-Object Planner, which can synthesize diverse hand-object trajectories; and (2) HOT, a Hand-Object Tracker that bridges synthetic-to-physical transfer through reinforcement learning and interaction imitation learning, delivering a generalizable controller conditioned on target hand-object states. Our method extends to diverse object shapes and hand morphologies. Through extensive evaluations, we show that our approach enables dexterous hands to track challenging, long-horizon sequences including object re-arrangement and agile in-hand reorientation. These results represent a significant step toward scalable foundation controllers for manipulation that can learn entirely from synthetic data, breaking the data bottleneck that has long constrained progress in dexterous manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。