用神经网络把人动作转成机器人能跑的高质量动作,避免关节突变和穿模。
Make Tracking Easy: Neural Motion Retargeting for Humanoid Whole-body Control
- 将动作重定向看作学习数据分布,而非传统优化,避免局部最优
- 通过聚类分组人类动作,提升训练效率并修复噪声示范
- 生成动作可加速机器人控制策略收敛,适合复杂任务部署
类人机器人需具备多样运动技能以适应复杂环境,但如何从人类数据中桥接运动学与动力学上的本体差异仍是主要瓶颈。我们通过海森分析表明,传统基于优化的重定向本质上非凸,易陷入局部最优,导致关节突变和自穿透等物理伪影。为此,我们将重定向问题重构为学习数据分布,提出神经动作重定向(NMR)框架,将静态几何映射转化为动态感知的学习过程。首先设计分簇专家物理精炼(CEPR),利用基于变分自编码器的动作聚类,将异构运动划分为潜在运动模式,显著降低大规模并行强化学习专家的计算开销;这些专家将噪声人类示范投影并修复至机器人可行运动流形上。由此生成的高保真数据监督一个非自回归的CNN-Transformer架构,能够全局建模时间上下文,抑制重建噪声并避开几何陷阱。在Unitree G1类人机器人上进行的多动态任务实验(如武术、舞蹈)显示,NMR有效消除关节跳变,显著减少自碰撞,且生成的动作参考可加速下游全身控制策略的收敛,为弥合人机本体差异提供了可扩展路径。
原文摘要 · Abstract (English)
Humanoid robots require diverse motor skills to integrate into complex environments, but bridging the kinematic and dynamic embodiment gap from human data remains a major bottleneck. We demonstrate through Hessian analysis that traditional optimization-based retargeting is inherently non-convex and prone to local optima, leading to physical artifacts like joint jumps and self-penetration. To address this, we reformulate the targeting problem as learning data distribution rather than optimizing optimal solutions, where we propose NMR, a Neural Motion Retargeting framework that transforms static geometric mapping into a dynamics-aware learned process. We first propose Clustered-Expert Physics Refinement (CEPR), a hierarchical data pipeline that leverages VAE-based motion clustering to group heterogeneous movements into latent motifs. This strategy significantly reduces the computational overhead of massively parallel reinforcement learning experts, which project and repair noisy human demonstrations onto the robot's feasible motion manifold. The resulting high-fidelity data supervises a non-autoregressive CNN-Transformer architecture that reasons over global temporal context to suppress reconstruction noise and bypass geometric traps. Experiments on the Unitree G1 humanoid across diverse dynamic tasks (e.g., martial arts, dancing) show that NMR eliminates joint jumps and significantly reduces self-collisions compared to state-of-the-art baselines. Furthermore, NMR-generated references accelerate the convergence of downstream whole-body control policies, establishing a scalable path for bridging the human-robot embodiment gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。