用强化学习辅助模型预测控制，让机器人自动生成稳定步态。
RL-Augmented MPC for Non-Gaited Legged and Hybrid Locomotion
- 高层强化学习生成步态指令，底层模型预测控制执行运动
- 仅需少量奖励设计即在多平台实现零样本仿真到仿真迁移
- 适用于轮腿混合机器人，真实世界部署无需额外训练
我们提出一种接触显式分层架构，将强化学习（RL）与模型预测控制（MPC）结合：高层RL代理提供步态和导航指令，低层运动控制采用MPC执行。该方法通过仿真中试错学习无循环步态，将接触时机的组合复杂性从MPC中剥离。实验表明，仅需极简奖励设计与有限调参即可获得有效策略。我们在50~120公斤级不同机器人平台上验证该架构，涵盖足式与混合式运动，在平坦地形上观察到无循环步态与接触时机自适应现象，并进一步拓展至非平坦地形。所有平台均实现无需领域随机化的零样本仿真到仿真迁移；并在120公斤级轮腿人形机器人Centauro上成功完成零样本仿真到真实世界的迁移。代码与评估结果已公开于https://github.com/AndrePatri/AugMPC。
原文摘要 · Abstract (English)
We propose a contact-explicit hierarchical architecture coupling Reinforcement Learning (RL) and Model Predictive Control (MPC), where a high-level RL agent provides gait and navigation commands to a low-level locomotion MPC. This offloads the combinatorial burden of contact timing from the MPC by learning acyclic gaits through trial and error in simulation. We show that only a minimal set of rewards and limited tuning are required to obtain effective policies. We validate the architecture in simulation across robotic platforms spanning 50 kg to 120 kg and different MPC implementations, observing the emergence of acyclic gaits and timing adaptations in flat-terrain legged and hybrid locomotion, and further demonstrating extensibility to non-flat terrains. Across all platforms, we achieve zero-shot sim-to-sim transfer without domain randomization, and we further demonstrate zero-shot sim-to-real transfer without domain randomization on Centauro, our 120 kg wheeled-legged humanoid robot. We make our software framework and evaluation results publicly available at https://github.com/AndrePatri/AugMPC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。