用强化学习增强模型预测控制，让双足机器人在复杂地形更稳健
RL-augmented Adaptive Model Predictive Control for Bipedal Locomotion over Challenging Terrain
- 用RL动态调整MPC的三要素：动力学、摆动腿控制和步频
- 在台阶、石块、低摩擦地面上表现优于传统MPC和纯RL方法
- 适合研究双足机器人运动控制或智能系统鲁棒性设计的人参考
模型预测控制（MPC）在类人双足行走中已证明有效，但在崎岖、湿滑等复杂地形上受限于难以建模地形交互。相比之下，强化学习（RL）在多样化地形上训练出鲁棒行走策略，但缺乏约束满足保证且常需大量奖励设计。近期融合MPC与RL的研究虽有进展，但多局限于平坦地形或四足机器人。本文提出一种面向双足机器人在粗糙、湿滑地形上的RL增强型MPC框架。方法通过参数化基于单刚体动力学的MPC三大组件：系统动力学、摆动腿控制器和步频。在NVIDIA IsaacLab仿真环境中验证了多种地形下的表现，包括台阶、踏石及低摩擦表面。结果表明，所提框架生成的行为显著更具适应性和鲁棒性，优于基线MPC与纯RL方法。
原文摘要 · Abstract (English)
Model predictive control (MPC) has demonstrated effectiveness for humanoid bipedal locomotion; however, its applicability in challenging environments, such as rough and slippery terrain, is limited by the difficulty of modeling terrain interactions. In contrast, reinforcement learning (RL) has achieved notable success in training robust locomotion policies over diverse terrain, yet it lacks guarantees of constraint satisfaction and often requires substantial reward shaping. Recent efforts in combining MPC and RL have shown promise of taking the best of both worlds, but they are primarily restricted to flat terrain or quadrupedal robots. In this work, we propose an RL-augmented MPC framework tailored for bipedal locomotion over rough and slippery terrain. Our method parametrizes three key components of single-rigid-body-dynamics-based MPC: system dynamics, swing leg controller, and gait frequency. We validate our approach through bipedal robot simulations in NVIDIA IsaacLab across various terrains, including stairs, stepping stones, and low-friction surfaces. Experimental results demonstrate that our RL-augmented MPC framework produces significantly more adaptive and robust behaviors compared to baseline MPC and RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。