无需中间转换,直接用语言控制人形机器人行走
From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance
- 用语言引导的运动隐空间直接生成机器人动作
- 实测延迟降低,成功率与追踪精度显著提升
- 支持文本、图像、音频等多模态输入
自然语言为人形机器人提供了直观的交互接口,但现有语言引导的行走系统流程繁琐且不可靠。传统方法需解码人类动作、重定向至机器人结构,再通过物理控制器跟踪,此多阶段过程易累积误差、引入高延迟,并削弱语义与控制的关联性。为此,我们提出 RoboGhost 框架,直接以语言锚定的运动隐空间为条件,生成人形机器人动作,跳过显式动作解码与重定向环节。该框架利用扩散模型从噪声中去噪生成可执行动作,保持语义一致性并实现快速响应。结合因果变换器与扩散模型的混合生成器,确保长时程动作连贯性,同时维持稳定性和多样性,生成丰富隐表示以精准控制人形行为。大量实验表明,RoboGhost 显著降低部署延迟,提升成功率与追踪精度,在真实人形机器人上实现平滑、语义对齐的行走。该框架还可自然扩展至图像、音频、音乐等其他模态,为视觉-语言-动作一体化人形系统提供通用基础。
原文摘要 · Abstract (English)
Natural language offers a natural interface for humanoid robots, but existing language-guided humanoid locomotion pipelines remain cumbersome and untrustworthy. They typically decode human motion, retarget it to robot morphology, and then track it with a physics-based controller. However, this multi-stage process is prone to cumulative errors, introduces high latency, and yields weak coupling between semantics and control. These limitations call for a more direct pathway from language to action, one that eliminates fragile intermediate stages. Therefore, we present RoboGhost, a retargeting-free framework that directly conditions humanoid policies on language-grounded motion latents. By bypassing explicit motion decoding and retargeting, RoboGhost enables a diffusion-based policy to denoise executable actions directly from noise, preserving semantic intent and supporting fast, reactive control. A hybrid causal transformer-diffusion motion generator further ensures long-horizon consistency while maintaining stability and diversity, yielding rich latent representations for precise humanoid behavior. Extensive experiments demonstrate that RoboGhost substantially reduces deployment latency, improves success rates and tracking precision, and produces smooth, semantically aligned locomotion on real humanoids. Beyond text, the framework naturally extends to other modalities such as images, audio, and music, providing a universal foundation for vision-language-action humanoid systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。