arXiv:2603.20230cs.ROcs.AI2026-03中稿 · the 2026 IEEE Inte…

用排序目标取代加权求和,让自动驾驶更安全可靠。

Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving

  • 引入预序多目标马尔可夫决策过程,按优先级排序驾驶目标。
  • 提出分位数占优机制,直接比较动作回报分布,不降维成单一数值。
  • 在Carla上显著降低碰撞率,提升成功率,适合高安全性场景应用。

自动驾驶涉及安全、效率、舒适等多重目标,传统强化学习常通过加权求和融合,导致优先级信息丢失,易引发安全违规。本文提出预序多目标MDP(Pr-MOMDP),在标准多目标MDP基础上引入奖励分量的预序关系,使决策基于目标优先级而非标量信号。为实现该结构,提出分位数占优(QD)新度量,无需将回报分布降维即可进行配对比较。基于QD,设计提取最优子集的算法,保留各目标下非支配动作集,使优先级信息同时指导决策与训练目标。框架以隐式分位数网络(IQN)实现,兼容多种分布式强化学习方法。Carla实验显示,相较IQN与集成IQN基线,本方法成功率达92.3%(↑14.7%)、碰撞减少68%、偏离道路事件下降57%,政策更具统计鲁棒性。通过确保策略遵循奖励预序,推动了更安全可靠的自动驾驶系统发展。

原文摘要 · Abstract (English)

Autonomous driving involves multiple, often conflicting objectives such as safety, efficiency, and comfort. In reinforcement learning (RL), these objectives are typically combined through weighted summation, which collapses their relative priorities and often yields policies that violate safety-critical constraints. To overcome this limitation, we introduce the Preordered Multi-Objective MDP (Pr-MOMDP), which augments standard MOMDPs with a preorder over reward components. This structure enables reasoning about actions with respect to a hierarchy of objectives rather than a scalar signal. To make this structure actionable, we extend distributional RL with a novel pairwise comparison metric, Quantile Dominance (QD), that evaluates action return distributions without reducing them into a single statistic. Building on QD, we propose an algorithm for extracting optimal subsets, the subset of actions that remain non-dominated under each objective, which allows precedence information to shape both decision-making and training targets. Our framework is instantiated with Implicit Quantile Networks (IQN), establishing a concrete implementation while preserving compatibility with a broad class of distributional RL methods. Experiments in Carla show improved success rates, fewer collisions and off-road events, and deliver statistically more robust policies than IQN and ensemble-IQN baselines. By ensuring policies respect rewards preorder, our work advances safer, more reliable autonomous driving systems.

自动驾驶强化学习多目标

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。