arXiv:2607.03177cs.SEcs.AI2026-07

用因果关系指导强化学习,让自动驾驶系统更好配合规则式恢复机制。

CRRL: A Causality-Based Reinforcement Learning Framework for Autonomous System Recovery

论文配图:CRRL: A Causality-Based Reinforcement Learning Framework for Autonomous System Recovery
图 1 · 摘自论文原文
  • 基于驾驶日志构建因果模型,指导策略提前预判故障并调整行为。
  • 在3个场景中,因果训练使奖励、行驶距离和速度显著提升,9/20次绕行无需干预。
  • 适合需要安全恢复机制的自动驾驶系统研究者,尤其关注可协作性强化学习。

传统强化学习在自动驾驶系统恢复中缺乏因果理解,泛化能力差,策略常在故障状态停滞,最多耗时70%的回合。仅依赖规则恢复不足,且向预训练PPO策略添加启发式恢复会恶化性能,因策略无法与突发干预协同。问题不在于缺少恢复手段,而在于策略未训练如何与之协作。本文提出CRRL,一种基于因果的强化学习框架,使策略能有效配合规则式恢复。当检测到停滞状态时,恢复模块介入辅助。利用驾驶日志中的因果关系构建训练信号,教会策略在恢复情境中提前调整动作。框架遵循MAPE-K范式:传感器采集、因果建模与混合强化学习训练分别对应监控、分析与规划/执行。我们在三个驾驶场景下进行四条件消融实验,每条件20个回合。结果表明,因果训练显著提升奖励、行驶距离与速度;9/20次绕行无需恢复干预,证明导航能力。说明因果引导训练可生成与规则安全组件协同有效的强化学习策略。

原文摘要 · Abstract (English)

Traditional reinforcement learning (RL) for recovery in autonomous systems lacks causal understanding and generalizes poorly to novel failure scenarios. RL policies often stall in failure states, spending up to 70% of an episode immobilized. Rule-based recovery alone is inadequate, and adding heuristic recovery to a pretrained PPO policy worsens rewards because policies cannot coordinate well with unanticipated interventions. The issue is not missing recovery mechanisms but a lack of policies trained to collaborate with them. We introduce CRRL, a causal-guided RL framework that trains policies to work effectively with rule-based recovery. The recovery detects stalled states and assists the agent. Causal relations from driving logs shape the training signal, teaching the policy to anticipate stalls and adjust actions in recovery contexts. The framework follows MAPE-K, with sensor collection, causal model construction, and hybrid RL policy training corresponding to Monitor, Analyze, and Plan/Execute, respectively. We evaluate CRRL through a four-condition ablation study across three driving scenarios, with 20 episodes per condition. We find that causal training significantly improves reward, distance, and velocity. Moreover, 9 of 20 roundabout episodes required zero recovery intervention, confirming navigation competence. These results show that causal-guided training produces effective RL policies that cooperate with rule-based safety components.

强化学习自动驾驶因果推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。