arXiv:2605.30576cs.AI2026-05中稿 · The IEEE Internati…

用不确定性触发专家指导,让自动驾驶强化学习更安全高效

Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving

论文配图:Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
图 1 · 摘自论文原文
  • 根据认知与随机不确定性动态触发专家建议
  • 成功率提升5-7%,事故率显著降低
  • 适合需要安全探索的自动驾驶强化学习场景

自动驾驶的强化学习中,探索行为固有风险:智能体需尝试新动作以学习,但可能引发碰撞或偏离道路。本文提出一种基于不确定性的引导框架,当认知或随机不确定性超过自适应阈值(由滑动缓冲区计算)时触发专家建议,确保指导随智能体信心变化而调整。采用承诺-冷却策略结合随机早停机制,调控指导持续时间与频率,避免过度依赖且保持动作连贯性。专家与智能体经验共享于离线策略隐式分位数网络(IQN)的回放缓冲区中,实现专家轨迹高效复用。在CARLA环境下的实验表明,相比IQN基线,本方法成功率提升5-7%,失败次数减少,验证了风险敏感的不确定性感知与受控专家融合可实现更安全高效的传感器驱动强化学习探索,适用于无信号交叉口导航任务。

原文摘要 · Abstract (English)

Exploration in reinforcement learning for autonomous driving is inherently unsafe: agents must experience novel behaviors to learn, yet exploration can lead to collisions or off-road driving. We propose an uncertainty-aware framework that leverages expert advice to guide exploration while avoiding long-term dependence. Advice is triggered when epistemic or aleatoric uncertainty exceeds adaptive thresholds derived from rolling buffers, ensuring advice evolves with the agent's confidence. A commitment-cooldown strategy with a stochastic early-stop heuristic regulates the duration and frequency of guidance, exposing the agent to coherent maneuvers without exhausting the advice budget. Expert and agent experiences are combined in a shared replay buffer within an off-policy implicit quantile network (IQN) backbone, enabling efficient reuse of expert trajectories. Experiments in CARLA show that our method outperforms the IQN baseline, improving success by 5-7% and reducing failures, demonstrating that risk-sensitive uncertainty coupled with regulated expert integration enables safer and more efficient exploration for sensor-based RL policy learning in unsignalized intersection navigation.

强化学习自动驾驶不确定性建模专家指导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。