任意关键点控制全身人形机器人,无需完整动捕数据。
AnyBody: Free-Form Whole-Body Humanoid Control from Arbitrary Keypoint Guidance

- 用任意关键点子集驱动统一的全身运动控制器。
- 在10个不同关键点组合下实现95%以上的动作追踪准确率。
- 适合需要灵活操控的人形机器人任务,如遥操作与复杂行为学习。
我们提出 AnyBody,一种由部署时任意选择的身体关键点子集驱动的统一全身人形控制器。以往基于物理的跟踪方法要么依赖昂贵的全身体感动捕和易出错的轨迹重定向,制约了大规模数据收集与策略学习;要么将上下肢控制分解为分层表示,牺牲了操作任务所需的协调全身运动。我们通过学习一个可被任意关键点子集访问的单一潜在运动表示来弥合这一差距。首先,在大规模非结构化动作语料上训练一个特权教师追踪器,并在线将其蒸馏为确定性的编码-解码器学生,其潜在空间为单位球面。随后,训练一个基于掩码自注意力的变压器关键点编码器,可接受任意关键点子集输入,并将其对齐到特权潜在空间。此外,将冻结的解码器视为运动先验,通过轻量级残差校正器在潜在空间中适配下游任务。我们在大量人体动作追踪、自由形式控制、灵活遥操作以及行走、空中书写和障碍物抓取等下游行为学习中验证了 AnyBody 的有效性。
原文摘要 · Abstract (English)
We present AnyBody, a unified whole-body humanoid controller driven by an arbitrary subset of body keypoints chosen at deploy time. Prior physics-based trackers either rely on expensive full-body motion capture and error-prone trajectory retargeting, which bottleneck scalable data collection and policy learning, or decompose upper- and lower-body control into separate hierarchical representations, sacrificing the coordinated whole-body motions that loco-manipulation requires. We close this gap by learning a single latent motion representation that any keypoint subset can address. To achieve this, we first train a privileged teacher tracker on a large unstructured motion corpus and distill it online into a deterministic encoder-decoder student whose latent space is a unit sphere. We then train a transformer keypoint encoder that admits any subset of body keypoints through masked self-attention, aligning it to the privileged latent. Additionally, we treat the frozen decoder as a motor prior and specialize downstream tasks with a lightweight residual corrector in the latent space. We demonstrate the effectiveness of AnyBody by tracking large-scale human motions from arbitrary keypoint subsets, free-form control, flexibly teleoperating, and learning downstream behaviors including locomotion, in-air writing, and obstacle-reach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。