提出谐波梯度方法，让自动驾驶在多车道场景中兼顾高效与安全。
Distributional Soft Actor-Critic with Harmonic Gradient for Safe and Efficient Autonomous Driving in Multi-lane Scenarios
- 通过计算效率与安全双梯度，生成谐波梯度优化策略更新
- 在多车道仿真中实现接近零的安全违规，驾驶效率高
- 适合需要高安全性的自动驾驶系统训练
强化学习（RL）因其自进化能力，为高层自动驾驶系统训练提供了前景。然而，现有算法在处理约束方面仍面临挑战，尤其在真实应用中。本文提出一种面向安全的训练技术——谐波策略迭代（HPI）。每轮RL迭代中，先分别计算高效驾驶与安全约束相关的两个策略梯度，再构造谐波梯度用于策略更新，以最小化两梯度间的冲突，实现更平衡稳定的训练过程。此外，我们采用最先进的DSAC算法作为基础框架，结合HPI构建新的安全强化学习算法DSAC-H。大量多车道场景仿真结果表明，DSAC-H在保持高效驾驶性能的同时，实现了近乎零的安全约束违规。
原文摘要 · Abstract (English)
Reinforcement learning (RL), known for its self-evolution capability, offers a promising approach to training high-level autonomous driving systems. However, handling constraints remains a significant challenge for existing RL algorithms, particularly in real-world applications. In this paper, we propose a new safety-oriented training technique called harmonic policy iteration (HPI). At each RL iteration, it first calculates two policy gradients associated with efficient driving and safety constraints, respectively. Then, a harmonic gradient is derived for policy updating, minimizing conflicts between the two gradients and consequently enabling a more balanced and stable training process. Furthermore, we adopt the state-of-the-art DSAC algorithm as the backbone and integrate it with our HPI to develop a new safe RL algorithm, DSAC-H. Extensive simulations in multi-lane scenarios demonstrate that DSAC-H achieves efficient driving performance with near-zero safety constraint violations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。