arXiv:2605.22894cs.GRcs.LG2026-05被引 2

用多阶段训练让机器人听懂指令并稳定执行复杂动作

SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control

论文配图:SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control
图 1 · 摘自论文原文
  • 用联合动作-状态-文本扩散模型,让语言与动作直接交互
  • 在1200小时数据上训练,性能随规模提升,长期控制更稳定
  • 适合研究具身智能、人形机器人控制的开发者和研究者

从自然语言指令控制物理模拟人形机器人是实现通用具身智能的关键一步。现有方法在语义表达与物理可行性间存在矛盾,难以同时实现精准指令跟随、高质量运动和稳定长时控制。本文提出SCRIPT,一种基于多阶段训练框架的可扩展扩散策略。核心为联合动作-状态-文本扩散变换器(JAST-DiT),将动作、物理状态和文本分别表示为独立令牌流,并通过联合注意力机制实现语言语义与控制动态的直接交互。为稳定自回归控制,引入非线性历史条件机制,保留密集近期上下文,逐步稀疏采样远期历史信息。除监督模仿预训练外,还设计后训练阶段,采用混合奖励强化学习(RLHR)进一步优化性能:通过向生成流程注入可学习噪声,在闭环仿真中结合物理反馈与文本奖励,显著提升运动质量与指令遵循度。定量评估显示,SCRIPT在文本对齐、运动质量与物理真实性等指标上均优于现有最优方法。在1200小时的MotionMillion数据集上的缩放实验表明,模型性能随规模持续提升,验证了其在大规模预训练中的强大可扩展性。代码将公开以促进后续研究。

原文摘要 · Abstract (English)

Controlling physics-based humanoids from natural-language instructions is a critical step toward general-purpose embodied agents. However, existing methods remain constrained by a tension between semantic expressiveness and physical feasibility, often failing to jointly achieve faithful instruction following, high-quality motion, and stable long-horizon control. We propose SCRIPT, a scalable diffusion policy with a multi-stage training framework for language-driven physics-based humanoid control. The core of SCRIPT is a Joint Action-State-Text Diffusion Transformer (JAST-DiT), which represents actions, physical states, and text as dedicated token streams and couples them through joint attention, enabling direct interaction between language semantics and control dynamics. To stabilize autoregressive control, we introduce a nonlinear history conditioning mechanism, which preserves the dense recent context and samples increasingly sparse cues from long-term history. Beyond supervised imitation pre-training, we propose a post-training stage, further improving the performance using Reinforcement Learning with Hybrid Rewards (RLHR). By injecting learnable noise into the flow-sampling process, RLHR effectively improves motion quality and instruction following within closed-loop simulations using hybrid physical feedback and text rewards. Quantitative evaluations demonstrate that SCRIPT outperforms prior state-of-the-art methods, with gains across text alignment, motion quality, and physical realism metrics. Furthermore, scaling studies on the 1200-hour MotionMillion dataset demonstrate consistent performance gains with model scaling, highlighting SCRIPT's robust scalability for large-scale pre-training. Our code will be publicly available for future research.

人形机器人扩散模型语言控制强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。