让机器学习模型自动满足安全规则,无需依赖复杂验证。
SMLE: Safe Machine Learning via Embedded Overapproximation
- 用可验证的架构+投影梯度法训练,保证模型行为符合预设规则。
- 在回归和多标签分类任务中均实现100%属性满足率,性能媲美前后处理方案。
- 适合需严格合规的医疗、自动驾驶等高风险场景应用。
尽管机器学习与神经网络取得了显著进展,但在这些系统上提供形式化行为保证仍是开放难题,尤其在受监管或高安全性场景中至关重要。本文研究如何训练满足设计者指定输入-输出逻辑关系的可微分机器学习模型。由于现代神经网络的严格验证与合规性强制计算复杂度高,该任务极具挑战性。我们提出一种创新框架,包含三个核心组件:1)通用且简洁的架构,支持高效验证并具有保守语义;2)基于投影梯度法的严格训练算法;3)强反例搜索问题的形式化表达。所提方法对模型复杂度影响极小,可扩展至实际应用,生成具备完整属性满足保证的模型。我们在回归任务中基于线性不等式定义的性质,以及多标签分类中的互斥类别性质上进行了评估。结果表明,本方法在性能上优于仅在数据预处理或预测后处理阶段加入属性约束的基线方案。此外,本工作为后续研究开辟了多个新方向与改进空间。
原文摘要 · Abstract (English)
Despite the extent of recent advances in Machine Learning (ML) and Neural Networks, providing formal guarantees on the behavior of these systems is still an open problem, and a crucial requirement for their adoption in regulated or safety-critical scenarios. We consider the task of training differentiable ML models guaranteed to satisfy designer-chosen properties, stated as input-output implications. This is very challenging, due to the computational complexity of rigorously verifying and enforcing compliance in modern neural models. We provide an innovative approach based on three components: 1) a general, simple architecture enabling efficient verification with a conservative semantic; 2) a rigorous training algorithm based on the Projected Gradient Method; 3) a formulation of the problem of searching for strong counterexamples. The proposed framework, being only marginally affected by model complexity, scales well to practical applications, and produces models that provide full property satisfaction guarantees. We evaluate our approach on properties defined by linear inequalities in regression, and on mutually exclusive classes in multilabel classification. Our approach is competitive with a baseline that includes property enforcement during preprocessing, i.e. on the training data, as well as during postprocessing, i.e. on the model predictions. Finally, our contributions establish a framework that opens up multiple research directions and potential improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。