将机器学习失败纳入动态安全评估，提升自动驾驶系统可靠性
Incorporating Failure of Machine Learning in Dynamic Probabilistic Safety Assurance
- 用贝叶斯网络建模机器学习失效，融合动态置信度判断
- 在模拟车列系统中实现不确定性下的实时安全评估与自适应
- 适合关注高阶安全验证的自动驾驶研究者
机器学习模型越来越多地被用于自动驾驶等安全关键系统中，以实现实时决策。然而，其固有的不完美性引入了一类新故障：由运行数据与训练数据分布偏移引发的推理失败。传统依赖设计文档或代码的安全评估方法难以适用于从数据中学习行为的机器学习组件。近期提出的SafeML可动态检测此类分布偏移，并为机器学习组件的推理赋予置信度。在此基础上，本文提出一种概率化安全保证框架，将SafeML与贝叶斯网络（BNs）结合，将机器学习故障作为更广泛因果安全分析的一部分进行建模。该框架支持在不确定性下进行动态安全评估与系统自适应。我们在一个包含交通标志识别的模拟汽车车列系统上验证了该方法，结果表明显式建模机器学习故障对安全评估具有显著提升潜力。
原文摘要 · Abstract (English)
Machine Learning (ML) models are increasingly integrated into safety-critical systems, such as autonomous vehicle platooning, to enable real-time decision-making. However, their inherent imperfection introduces a new class of failure: reasoning failures often triggered by distributional shifts between operational and training data. Traditional safety assessment methods, which rely on design artefacts or code, are ill-suited for ML components that learn behaviour from data. SafeML was recently proposed to dynamically detect such shifts and assign confidence levels to the reasoning of ML-based components. Building on this, we introduce a probabilistic safety assurance framework that integrates SafeML with Bayesian Networks (BNs) to model ML failures as part of a broader causal safety analysis. This allows for dynamic safety evaluation and system adaptation under uncertainty. We demonstrate the approach on an simulated automotive platooning system with traffic sign recognition. The findings highlight the potential broader benefits of explicitly modelling ML failures in safety assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。