用扩散模型+在线强化学习,让四足机器人听懂指令、稳定换任务。
Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Control
- 先用扩散模型多任务预训练,支持语言指令控制
- 再在仿真中用PPO微调,确保真实世界稳定切换任务
- 模型可部署在50Hz的嵌入式设备上,适合资源受限平台
近期研究显示,模仿学习在机器人领域展现出强大能力。利用生成模型,尤其是扩散模型,这类方法具备出色的多任务泛化能力、有效的语言条件控制和高样本效率。尽管其在操作任务中已取得成功,但在腿式运动中的应用仍相对有限,主要受制于累积误差带来的稳定性问题以及数据稀缺下的任务切换困难。近年来,在线强化学习(RL)在腿式机器人控制中表现优异,为解决这些问题提供了新思路。本文提出DMLoco,一种基于扩散模型的四足机器人框架,通过多任务预训练与在线PPO微调相结合,实现语言条件控制与鲁棒的任务切换。首先,在多样化的多任务数据集上使用扩散模型预训练策略,实现多种技能的语言引导执行;随后,在仿真中进行PPO微调,确保真实部署时的鲁棒性与稳定任务过渡。通过采用去噪扩散隐式模型(DDIM)实现高效采样,并利用TensorRT优化部署,策略可在机载系统上以50Hz运行,为资源受限机器人平台提供了一种可扩展、高效的自适应语言驱动运动解决方案。
原文摘要 · Abstract (English)
Recent research has highlighted the powerful capabilities of imitation learning in robotics. Leveraging generative models, particularly diffusion models, these approaches offer notable advantages such as strong multi-task generalization, effective language conditioning, and high sample efficiency. While their application has been successful in manipulation tasks, their use in legged locomotion remains relatively underexplored, mainly due to compounding errors that affect stability and difficulties in task transition under limited data. Online reinforcement learning (RL) has demonstrated promising results in legged robot control in the past years, providing valuable insights to address these challenges. In this work, we propose DMLoco, a diffusion-based framework for quadruped robots that integrates multi-task pretraining with online PPO finetuning to enable language-conditioned control and robust task transitions. Our approach first pretrains the policy on a diverse multi-task dataset using diffusion models, enabling language-guided execution of various skills. Then, it finetunes the policy in simulation to ensure robustness and stable task transition during real-world deployment. By utilizing Denoising Diffusion Implicit Models (DDIM) for efficient sampling and TensorRT for optimized deployment, our policy runs onboard at 50Hz, offering a scalable and efficient solution for adaptive, language-guided locomotion on resource-constrained robotic platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。