提出统一框架,让自动驾驶在训练和部署中兼顾安全与效率。
Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency

- 融合安全距离、奖励机与专家混合模型,实现规则感知决策。
- 在双车道随机交通中,安全性和效率显著优于现有方法。
- 适合研究自动驾驶决策与安全强化学习的工程师和学者。
深度强化学习(DRL)为高级自动驾驶车辆提供了有前景的决策路径,但其试错特性使得训练过程难以保证安全,且部署时难以同时实现安全与高效。本文提出一种统一的安全强化学习(SRL)框架,结合安全距离(SD)、奖励机(RM)与专家混合模型(MoE),命名为MoE-RM-SRL。部署阶段,SD与RM协同构建规则感知奖励,编码高速路交通规则与分阶段目标,实现安全可靠行为而不牺牲效率。训练阶段引入稀疏门控的MoE层,包含最多11个深度Q网络(DQNs);基于安全距离的门控规则仅激活最小数量的专家用于变道与保持车道,缓解了异构控制器(如MPC/规则基模块与学习策略)切换带来的不稳定性、不连续性及突变响应。我们在CARLA中实现该架构,并集成6自由度驾驶员在环虚拟现实平台(DiL-VR)。双车道随机交通实验表明,MoE-RM-SRL显著优于当前最优基线,在安全性与效率上均有提升,且可自然扩展至多车道驾驶、匝道汇入与驶出等场景。
原文摘要 · Abstract (English)
Deep reinforcement learning (DRL) offers a compelling route to decision-making for advanced autonomous vehicles (AVs), yet its trial-and-error nature makes it difficult to guarantee safety during training and to achieve both safety and efficiency at deployment. We propose a unified safe reinforcement learning (SRL) framework that integrates safe distance (SD), reward machines (RM), and mixture-of-experts (MoE), termed MoE-RM-SRL. For deployment, SD and RM jointly shape a rule-aware reward that encodes highway traffic regulations and stage-wise objectives, enabling safe and reliable behavior without sacrificing efficiency. For training, we introduce a sparsely gated MoE layer comprising up to 11 deep Q-networks (DQNs); an SD-based gating rule activates a minimal set of experts for lane-keeping and lane-changing, mitigating the instability, discontinuities, and impulsive transients commonly induced by switching between heterogeneous controllers (e.g., MPC/rule-based modules and learned policies). We implement the proposed architecture in CARLA and integrate it with a 6-DoF driver-in-the-loop virtual-reality (DiL-VR) platform. Experiments in stochastic two-lane traffic show that MoE-RM-SRL substantially improves safety and efficiency over state-of-the-art baselines, and the framework naturally extends to multi-lane driving as well as on-ramp merging and exiting scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。