arXiv:2409.09441cs.ROcs.LG2024-09ICRA被引 11

融合本体感知与强化学习,实现四足机器人长时序安全运动规划。

PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion

  • 用内建速度估测器和Dreamer模块构建本体感知规划框架
  • 在多地形上比传统MPC更稳定,硬件测试验证鲁棒性
  • 适合需要长期规划与约束处理的机器人控制场景

模型预测控制(MPC)在四足机器人运动中擅长约束执行与命令可解释性,但难以应对复杂任务,尤其在快速变化地面上表现不佳。相反,无模型强化学习(RL)虽在多地形上表现更优,能生成复杂动作,却无法处理约束或进行有效规划。为此,我们提出PIP-Loco框架,将本体感知规划与强化学习结合,实现长时序、安全、敏捷的运动行为。受MPC启发,框架包含速度估测器和Dreamer模块。训练阶段,专家策略与内部模型协同学习,促进探索以提升运动性能;部署阶段,Dreamer模块求解无限时域MPC问题,动态调整动作与速度指令以满足约束。通过消融实验验证内部模型组件的有效性,并展示对训练噪声更强的鲁棒性。最终在仿真与真实硬件上评估多地形适应能力,结果表明该方法显著优于传统MPC。

原文摘要 · Abstract (English)

A core strength of Model Predictive Control (MPC) for quadrupedal locomotion has been its ability to enforce constraints and provide interpretability of the sequence of commands over the horizon. However, despite being able to plan, MPC struggles to scale with task complexity, often failing to achieve robust behavior on rapidly changing surfaces. On the other hand, model-free Reinforcement Learning (RL) methods have outperformed MPC on multiple terrains, showing emergent motions but inherently lack any ability to handle constraints or perform planning. To address these limitations, we propose a framework that integrates proprioceptive planning with RL, allowing for agile and safe locomotion behaviors through the horizon. Inspired by MPC, we incorporate an internal model that includes a velocity estimator and a Dreamer module. During training, the framework learns an expert policy and an internal model that are co-dependent, facilitating exploration for improved locomotion behaviors. During deployment, the Dreamer module solves an infinite-horizon MPC problem, adapting actions and velocity commands to respect the constraints. We validate the robustness of our training framework through ablation studies on internal model components and demonstrate improved robustness to training noise. Finally, we evaluate our approach across multi-terrain scenarios in both simulation and hardware.

四足机器人强化学习运动规划本体感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。