用强化学习让路侧单元协调自动驾驶车辆安全通过无信号交叉口。
Multi-Agent Reinforcement Learning-based Cooperative Autonomous Driving in Smart Intersections
- 路侧单元用两阶段强化学习预训练并微调决策模型。
- 三辆自动驾驶车通过复杂路口失败率低于0.03%。
- 适合智能交通、自动驾驶系统研发人员参考。
无信号交叉口因交通流复杂而带来显著的安全与效率挑战。本文提出一种基于路侧单元（RSU）的协同驾驶系统，利用全局感知与车-路通信实现高效控制。系统核心为基于RSU的决策模块，采用两阶段混合强化学习框架：首先在离线数据集上结合保守Q学习（CQL）与行为克隆（BC）预训练策略；随后在仿真中使用多智能体近端策略优化（MAPPO）进行微调，并引入自注意力机制有效处理智能体间依赖关系。RSU基于训练好的模型实时推理，通过车-路通信实现车辆控制。在CARLA环境中的大量实验表明，该系统具有高有效性：(i) 在复杂交叉口场景下，三辆联网自动驾驶车辆的协调失败率低于0.03%，显著优于传统Autoware控制方法；(ii) 在不同数量控制车辆及不同地图上均表现出强鲁棒性与良好泛化能力。
原文摘要 · Abstract (English)
Unsignalized intersections pose significant safety and efficiency challenges due to complex traffic flows. This paper proposes a novel roadside unit (RSU)-centric cooperative driving system leveraging global perception and vehicle-to-infrastructure (V2I) communication. The core of the system is an RSU-based decision-making module using a two-stage hybrid reinforcement learning (RL) framework. At first, policies are pre-trained offline using conservative Q-learning (CQL) combined with behavior cloning (BC) on collected dataset. Subsequently, these policies are fine-tuned in the simulation using multi-agent proximal policy optimization (MAPPO), aligned with a self-attention mechanism to effectively solve inter-agent dependencies. RSUs perform real-time inference based on the trained models to realize vehicle control via V2I communications. Extensive experiments in CARLA environment demonstrate high effectiveness of the proposed system, by: \textit{(i)} achieving failure rates below 0.03\% in coordinating three connected and autonomous vehicles (CAVs) through complex intersection scenarios, significantly outperforming the traditional Autoware control method, and \textit{(ii)} exhibiting strong robustness across varying numbers of controlled agents and shows promising generalization capabilities on other maps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。