用文字实时控制人形机器人,边说边动,自由切换动作
TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Control
- 分两层:上层用文本生成短时运动轨迹,下层实时执行到机器人
- 支持边运行边改指令,跳舞跳绳等复杂动作无缝衔接
- 适合做交互式机器人演示或实验,尤其看重自然语言控制
近期人形机器人全身运动追踪技术已能在真实硬件上实现多样且高度协调的动作。然而,现有控制器通常依赖预设运动轨迹,用户意图变化时灵活性差;或依赖持续的人类远程操作,限制了自主性。本文提出 TextOp,一种实时、可交互的文字驱动人形机器人运动生成与控制框架,支持流式语言指令及运行中动态修改。TextOp 采用两级架构:高层基于自回归的运动扩散模型,根据当前文本输入持续生成短时运动轨迹;低层运动追踪策略将这些轨迹在物理人形机器人上执行。通过连接交互式运动生成与鲁棒的全身控制,TextOp 实现自由形式意图表达,并在单次连续执行中平滑完成跳舞、跳跃等复杂行为。大量真实机器人实验与离线评估表明,系统具备即时响应、全身运动流畅、控制精准。项目主页与开源代码见 https://text-op.github.io/
原文摘要 · Abstract (English)
Recent advances in humanoid whole-body motion tracking have enabled the execution of diverse and highly coordinated motions on real hardware. However, existing controllers are commonly driven either by predefined motion trajectories, which offer limited flexibility when user intent changes, or by continuous human teleoperation, which requires constant human involvement and limits autonomy. This work addresses the problem of how to drive a universal humanoid controller in a real-time and interactive manner. We present TextOp, a real-time text-driven humanoid motion generation and control framework that supports streaming language commands and on-the-fly instruction modification during execution. TextOp adopts a two-level architecture in which a high-level autoregressive motion diffusion model continuously generates short-horizon kinematic trajectories conditioned on the current text input, while a low-level motion tracking policy executes these trajectories on a physical humanoid robot. By bridging interactive motion generation with robust whole-body control, TextOp unlocks free-form intent expression and enables smooth transitions across multiple challenging behaviors such as dancing and jumping, within a single continuous motion execution. Extensive real-robot experiments and offline evaluations demonstrate instant responsiveness, smooth whole-body motion, and precise control. The project page and the open-source code are available at https://text-op.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。