用强化学习提升MPC规划效率，实现装载机自主高效导航
Autonomous Wheel Loader Navigation Using Goal-Conditioned Actor-Critic MPC
- 将强化学习的批评网络融入MPC成本函数，直接增强规划能力
- 仿真与实机测试均显示路径生成速度优于传统轨迹优化方法
- 适合需要实时自主导航的工程机械领域研究者参考
本文提出一种新型控制方法，用于自主轮式装载机高效导航至任意目标位姿。不同于以往将高层轨迹规划器与模型预测控制（MPC）结合的方法，我们通过引入基于强化学习（RL）的代价函数，直接增强MPC的规划能力。具体而言，先在仿真中训练一个演员-评论家RL代理完成位姿到达任务，随后将训练好的神经网络评论家作为阶段成本和终端成本嵌入MPC。大量仿真实验表明，所提出的MPC继承了RL代理的时间高效特性，生成的轨迹优于传统轨迹优化方法。此外，该方法已在真实轮式装载机上部署，并在多种场景下成功实现导航。
原文摘要 · Abstract (English)
This paper proposes a novel control method for an autonomous wheel loader, enabling time-efficient navigation to an arbitrary goal pose. Unlike prior works which combine high-level trajectory planners with Model Predictive Control (MPC), we directly enhance the planning capabilities of MPC by incorporating a cost function derived from Actor-Critic Reinforcement Learning (RL). Specifically, we first train an RL agent to solve the pose reaching task in simulation, then transfer the learned planning knowledge to an MPC by incorporating the trained neural network critic as both the stage and terminal cost. We show through comprehensive simulations that the resulting MPC inherits the time-efficient behavior of the RL agent, generating trajectories that compare favorably against those found using trajectory optimization. We also deploy our method on a real-world wheel loader, where we demonstrate successful navigation in various scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。