arXiv:2606.16480cs.ROcs.AI2026-06

用分层策略让机器人在多场景下实时规划运动,无需每种场景重新调参。

HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization

论文配图:HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization
图 1 · 摘自论文原文
  • 高层策略离线学习抽象动作,生成鲁棒路径;低层在线用MPPI优化控制。
  • 在多种驾驶场景中,比传统MPPI和端到端强化学习更稳定且实时性高。
  • 适合需要跨场景自适应的机器人运动规划,如自动驾驶、服务机器人。

真实世界中的机器人需在多样场景中规划运动而无需针对每个场景重新调整。端到端强化学习虽具泛化能力,但在分布偏移、奖励设定错误和随机交互下易变得脆弱。模型预测路径积分(MPPI)控制无需梯度即可实现强实时优化,但其性能依赖于设计良好的采样先验,而手动设计先验难以扩展至多场景部署。本文提出HOLO-MPPI(高层离线、低层在线的MPPI),结合高层策略学习与低层随机最优控制。离线阶段,通过学习一个高层策略,在抽象动作空间中生成场景鲁棒的规划,并利用学习的世界模型进行在线回放。在线阶段,该策略作为数据驱动的先验生成器,根据当前观测和目标参数化MPPI的采样分布。随后,MPPI在实时内围绕此先验优化低层控制序列,以应对局部扰动。我们在自动驾驶中实现了HOLO-MPPI,设计了有效的高层动作空间和定制模型架构。在多种驾驶场景下的评估显示,其性能优于MPPI和端到端强化学习基线,同时保持实时控制能力。

原文摘要 · Abstract (English)

Robots deployed in the real world must plan motions across diverse scenarios without per-scenario retuning. End-to-end reinforcement learning (RL) can generalize across scenarios but often becomes brittle under distribution shift, reward misspecification, and stochastic interactions. Model predictive path integral (MPPI) control enables strong real-time refinement without gradients, but its performance depends on a well-shaped sampling prior, while manually designing the priors does not scale to multi-scenario deployment. We present HOLO-MPPI (High-level Offline, Low-level Online MPPI), a multi-scenario motion planning framework that combines high-level policy learning with low-level stochastic optimal control. Offline, we learn a high-level policy that proposes scenario-robust plans in an abstract action space, with a learned world model for online rollout. Online, the policy serves as a data-driven prior generator that parameterizes MPPI's sampling distribution conditioned on the current observation and goal. MPPI then optimizes low-level control sequences around this prior in real time to adapt to local disturbances. We instantiate HOLO-MPPI in autonomous driving by designing an effective high-level action space and tailored model architectures. Our evaluation across diverse driving scenarios shows that HOLO-MPPI improves upon MPPI and end-to-end RL baselines while maintaining real-time control.

运动规划强化学习自动驾驶实时控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。