通过动态选择专家策略,实现人形机器人高精度实时全身遥操作。
TeleGate: Whole-Body Humanoid Teleoperation via Gated Expert Selection with Motion Prior
- 用轻量门控网络实时选择特定领域专家策略,保留各动作最优性能。
- 仅用2.5小时动捕数据训练,实现跑步、跳跃等动态动作的高精度跟踪。
- 引入基于VAE的运动先验模块,预测未来动作意图,提升预判控制能力。
实时全身遥操作是人形机器人在非结构化环境中执行复杂任务的关键方法。然而,开发一个能稳健支持多种人类动作的统一控制器仍是重大挑战。现有方法通常将多个专家策略压缩为单一通用策略,常导致性能下降,尤其在高动态动作中更为明显。本文提出TeleGate,一种统一的人形机器人全身遥操作框架,可在多种动作下实现高精度跟踪,避免知识蒸馏带来的性能损失。核心思想是通过训练轻量级门控网络,在实时中根据本体感知状态和参考轨迹动态激活专家策略,以保留各领域专家的完整能力。此外,为弥补实时遥操作中缺乏未来参考轨迹的问题,我们引入基于变分自编码器(VAE)的运动先验模块,从历史观测中提取隐含的未来动作意图,实现对跳跃、起身等需预测动作的前瞻控制。我们在仿真中进行了实验,并将该方法部署于Unitree G1人形机器人上。仅使用2.5小时动作捕捉数据训练,TeleGate即可在多样化动态动作(如跑步、跌倒恢复、跳跃)中实现高精度实时遥操作,显著优于基线方法,在跟踪精度和成功率上均有大幅提升。
原文摘要 · Abstract (English)
Real-time whole-body teleoperation is a critical method for humanoid robots to perform complex tasks in unstructured environments. However, developing a unified controller that robustly supports diverse human motions remains a significant challenge. Existing methods typically distill multiple expert policies into a single general policy, which often inevitably leads to performance degradation, particularly on highly dynamic motions. This paper presents TeleGate, a unified whole-body teleoperation framework for humanoid robots that achieves high-precision tracking across various motions while avoiding the performance loss inherent in knowledge distillation. Our key idea is to preserve the full capability of domain-specific expert policies by training a lightweight gating network, which dynamically activates experts in real-time based on proprioceptive states and reference trajectories. Furthermore, to compensate for the absence of future reference trajectories in real-time teleoperation, we introduce a VAE-based motion prior module that extracts implicit future motion intent from historical observations, enabling anticipatory control for motions requiring prediction such as jumping and standing up. We conducted empirical evaluations in simulation and also deployed our technique on the Unitree G1 humanoid robot. Using only 2.5 hours of motion capture data for training, our TeleGate achieves high-precision real-time teleoperation across diverse dynamic motions (e.g., running, fall recovery, and jumping), significantly outperforming the baseline methods in both tracking accuracy and success rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。