用扩散模型生成未来轨迹,让一个智能体学会在各种地形上行走。
DreamPolicy: A Unified World-model Policy for Scalable Humanoid Locomotion
- 基于扩散模型构建地形感知的世界模型,自动生成未来运动轨迹。
- 在未见过的复合地形上表现优于基线27%,组合地形上提升38%。
- 适合研究通用人形机器人控制与数据驱动强化学习的开发者。
实现单一策略下的多样化人形机器人行走面临可扩展性挑战。现有方法通常将多个地形专用教师策略蒸馏为统一学生策略,但仅能捕捉基本运动模式,难以有机组合技能以适应复杂环境,导致对训练中未见的复合地形泛化能力差。为此,我们提出DreamPolicy,一种融合离线数据与基于扩散的世界模型的统一框架,使单个策略掌握已知及未见地形。核心是地形感知的世界模型,由专用策略的聚合轨迹训练的自回归扩散模型驱动,能合成物理合理的未来轨迹,作为条件策略的动态目标,避免人工奖励设计。相比蒸馏,该世界模型捕获可泛化的运动技能,实现对未见复合地形的鲁棒零样本迁移。DreamPolicy随数据量增长自然扩展:离线数据越多,扩散世界模型持续获取更丰富的技能。实验表明,其在未见地形上性能优于最强基线27%,在组合地形上提升38%。通过统一世界模型规划与策略学习,突破‘一任务一策略’瓶颈,建立可扩展、数据驱动的通用型人形控制范式。
原文摘要 · Abstract (English)
Achieving versatile humanoid locomotion with a single policy presents a critical scalability challenge. Prevailing methods often rely on distilling multiple terrain-specific teacher policies into a unified student policy. However, while such distillation captures basic locomotion primitives, it struggles to organically compose these skills to adapt to complex environments, resulting in poor generalization to novel composite terrains unseen during training. To overcome this, we present DreamPolicy, a unified framework that integrates offline data with a diffusion-based world model, enabling a single policy to master both known and unseen terrains. Central to our approach is a terrain-aware world model, driven by an autoregressive diffusion world model trained on aggregated rollouts from specialized policies. This model synthesizes physically plausible future trajectories, which serve as dynamic objectives for a conditioned policy, thereby bypassing manual reward engineering. Unlike distillation, our world model captures generalizable locomotion skills, allowing for robust zero-shot transfer to unseen composite terrains. DreamPolicy naturally scales with data availability. As the offline dataset expands, the diffusion world model continuously acquires richer skills. Experiments demonstrate that DreamPolicy outperforms the strongest baseline by up to 27\% on unseen terrains and 38\% on combined terrains. By unifying world model-based planning and policy learning, DreamPolicy breaks the "one task, one policy" bottleneck and establishes a scalable, data-driven paradigm for generalist humanoid control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。