用自适应修复提升自动驾驶安全,让系统更聪明地应对危险场景。
ADReFT: Adaptive Decision Repair for Safe Autonomous Driving via Reinforcement Fine-Tuning
- 通过双头变压器模型识别危险状态并生成灵活修复动作。
- 离线预训练+强化微调,修复成功率更高且更少过度保守。
- 适合关注自动驾驶安全性与决策鲁棒性的研究者与工程师。
自动驾驶系统因设计与性能局限仍面临严重安全风险。在线修复在保障运行安全中起关键作用,现有方法多依赖预设规则或数据集将不可接受轨迹转为可接受,但普遍存在泛化性差、适应性弱和过于保守的问题,导致修复效果不佳且影响驾驶体验。为此,我们提出自适应决策修复(ADReFT),通过离线学习失败测试样本,识别安全关键状态,并生成合适缓解动作以提升系统安全性。ADReFT采用基于变压器的双头模型,包含状态监测器与决策适配器,捕捉复杂交通环境交互,评估状态安全严重性并生成自适应修复策略。由于缺乏安全状态标识的真值,我们首先使用粗标注(违规前状态为正样本,其余为负样本)进行监督预训练,建立基本修复能力,但可能较保守;随后通过强化学习微调,优化初始能力,生成更精准、上下文相关的修复决策。实验结果表明,ADReFT在修复性能上表现更优。
原文摘要 · Abstract (English)
Autonomous Driving Systems (ADSs) continue to face safety-critical risks due to the inherent limitations in their design and performance capabilities. Online repair plays a crucial role in mitigating such limitations, ensuring the runtime safety and reliability of ADSs. Existing online repair solutions enforce ADS compliance by transforming unacceptable trajectories into acceptable ones based on predefined specifications, such as rule-based constraints or training datasets. However, these approaches often lack generalizability, adaptability and tend to be overly conservative, resulting in ineffective repairs that not only fail to mitigate safety risks sufficiently but also degrade the overall driving experience. To address this issue, we propose Adaptive Decision Repair (ADReFT), a novel and effective repair method that identifies safety-critical states through offline learning from failed tests and generates appropriate mitigation actions to improve ADS safety. Specifically, ADReFT incorporates a transformer-based model with two joint heads, State Monitor and Decision Adapter, designed to capture complex driving environment interactions to evaluate state safety severity and generate adaptive repair actions. Given the absence of oracles for state safety identification, we first pretrain ADReFT using supervised learning with coarse annotations, i.e., labeling states preceding violations as positive samples and others as negative samples. It establishes ADReFT's foundational capability to mitigate safety-critical violations, though it may result in somewhat conservative mitigation strategies. Therefore, we subsequently finetune ADReFT using reinforcement learning to improve its initial capability and generate more precise and contextually appropriate repair decisions. Our evaluation results illustrate that ADReFT achieves better repair performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。