融合MPC与强化学习,让自动驾驶在复杂路口更安全高效。
Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning

- 用MPC提供约束框架,结合深度强化学习实现自适应决策
- 碰撞率降21%,成功率提升6.5%,优于纯MPC和端到端强化学习
- MPC组件提升跨场景泛化能力,适合追求安全与效率平衡的场景
无信号交叉口的自动驾驶面临多车交互复杂、安全与效率难以兼顾的挑战。模型预测控制(MPC)通过优化处理约束,但依赖人工设计规则,常导致过度保守;深度强化学习(RL)可从经验中学习适应性行为,却难保证安全且泛化能力差。本文提出一种集成MPC-RL框架,以提升多智能体场景下的导航性能。实验表明,该框架在三种交通密度下均优于独立使用MPC或端到端强化学习,相较纯MPC降低21%碰撞率,成功率提升6.5%。进一步零样本迁移至高速公路汇入场景,基于MPC的方法显著优于端到端PPO,凸显了MPC主干对跨场景鲁棒性的贡献。训练过程中,该框架损失收敛更快,表明学习负担更低。结果表明,该方法能更好平衡安全与效率,且MPC组件为不同驾驶环境的泛化提供了坚实基础。实现代码已开源。
原文摘要 · Abstract (English)
Automated driving at unsignalized intersections is challenging due to complex multi-vehicle interactions and the need to balance safety and efficiency. Model Predictive Control (MPC) offers structured constraint handling through optimization but relies on hand-crafted rules that often produce overly conservative behavior. Deep Reinforcement Learning (RL) learns adaptive behaviors from experience but often struggles with safety assurance and generalization to unseen environments. In this study, we present an integrated MPC-RL framework to improve navigation performance in multi-agent scenarios. Experiments show that MPC-RL outperforms standalone MPC and end-to-end RL across three traffic-density levels. Collectively, MPC-RL reduces the collision rate by 21% and improves the success rate by 6.5% compared to pure MPC. We further evaluate zero-shot transfer to a highway merging scenario without retraining. Both MPC-based methods transfer substantially better than end-to-end PPO, which highlights the role of the MPC backbone in cross-scenario robustness. The framework also shows faster loss stabilization than end-to-end RL during training, which indicates a reduced learning burden. These results suggest that the integrated approach can improve the balance between safety performance and efficiency in multi-agent intersection scenarios, while the MPC component provides a strong foundation for generalization across driving environments. The implementation code is available open-source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。