让机器人从人视频学精细操作，更稳更准。
ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling
- 用人体手臂轨迹+几何约束优化运动路径
- 多指接触用多面体建模，保持抓握稳定
- 实机测试成功率达90%以上，适配不同物体
从人类操作视频学习机器人灵巧操作，需可靠地将人类意图转化为可执行的机器人动作，同时保持手与物体的稳定接触，这仍是具身智能中的关键挑战。现有重定向方法常忽略显式接触建模或依赖强化学习，导致准确率和泛化能力有限。为此，我们提出ObjRetarget，一种从人类视频学习机器人灵巧操作的人-机运动重定向框架，融合类人手臂轨迹约束与结构化手-物几何建模。对于手臂运动，以人类视频中提取的参考轨迹为初始，结合类人约束与冗余感知优化，生成自然且精准的动作。对于手部操作，使用多面体簇表示多指接触，并通过几何不变量保留接触结构，提升稳定性。在真实机器人上的实验表明，ObjRetarget在多个灵巧任务中提升了操作成功率与接触稳定性，并能良好泛化至不同演示、物体姿态及任务设置。
原文摘要 · Abstract (English)
Learning robot dexterous manipulation from human manipulation videos requires reliably retargeting human intent to executable robot actions while maintaining stable hand-object contact, which remains a key challenge in embodied intelligence. Existing retargeting methods often ignore explicit contact modeling or rely on reinforcement learning, resulting in limited accuracy and generalization. To address this, we propose ObjRetarget, a human-to-robot motion retargeting framework for learning robot dexterous manipulation from human videos, which integrates anthropomorphic arm trajectory constraints with structured hand-object geometric modeling. For arm motion, reference trajectories extracted from human videos are used for initialization, followed by anthropomorphic constraints and redundancy-aware optimization to generate natural and accurate movements. For hand manipulation, ObjRetarget represents multi-finger contacts using polytope clusters and preserves contact structure through geometric invariants to improve stability. Experiments on real robots show that ObjRetarget improves manipulation success rates and contact stability across multiple dexterous tasks, and generalizes well to different demonstrations, object poses, and task settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。