提出可验证的定向解释框架,让AI决策更可信且聚焦关键风险。
Towards Verified and Targeted Explanations through Formal Methods
- 基于形式化方法,识别影响特定误判的关键特征子集。
- 在扰动不超过epsilon时,数学证明模型不会误判为目标类别。
- 适合自动驾驶、医疗诊断等高风险场景的可解释性需求。
随着深度神经网络在自动驾驶、医疗诊断等安全关键领域应用,利益相关方需要既可理解又具数学保证的解释。现有XAI方法存在局限:启发式归因技术(如LIME、Integrated Gradients)虽能定位关键特征,但无法保证决策边界;形式化方法可验证鲁棒性,却缺乏针对性,分析最近边界而不考虑实际风险。在安全关键系统中,并非所有误判后果相同——将“停止”标志误认为“60公里/小时”标志,远比误认为“禁止超车”标志更危险。本文提出ViTaX(Verified and Targeted Explanations)框架,生成具有数学保证的定向半反事实解释。针对给定输入(类别y)和用户指定的临界替代类别(类别t),ViTaX:(1) 找出最敏感于y→t转换的最小特征子集;(2) 利用形式化可达性分析,确保这些特征扰动不超过ε时,分类结果不会变为t。通过定义目标ε-鲁棒性,证明特征子集在特定目标方向上的抗扰能力。ViTaX是首个提供对用户指定替代类别的形式化保障解释的方法。在MNIST、GTSRB、EMNIST和TaxiNet上的实验表明,其解释保真度提升超过30%,且解释规模极小。
原文摘要 · Abstract (English)
As deep neural networks are deployed in safety-critical domains such as autonomous driving and medical diagnosis, stakeholders need explanations that are interpretable but also trustworthy with formal guarantees. Existing XAI methods fall short: heuristic attribution techniques (e.g., LIME, Integrated Gradients) highlight influential features but offer no mathematical guarantees about decision boundaries, while formal methods verify robustness yet remain untargeted, analyzing the nearest boundary regardless of whether it represents a critical risk. In safety-critical systems, not all misclassifications carry equal consequences; confusing a "Stop" sign for a "60 kph" sign is far more dangerous than confusing it with a "No Passing" sign. We introduce ViTaX (Verified and Targeted Explanations), a formal XAI framework that generates targeted semifactual explanations with mathematical guarantees. For a given input (class y) and a user-specified critical alternative (class t), ViTaX: (1) identifies the minimal feature subset most sensitive to the y->t transition, and (2) applies formal reachability analysis to guarantee that perturbing these features by epsilon cannot flip the classification to t. We formalize this through Targeted epsilon-Robustness, certifying whether a feature subset remains robust under perturbation toward a specific target class. ViTaX is the first method to provide formally guaranteed explanations of a model's resilience against user-identified alternatives. Evaluations on MNIST, GTSRB, EMNIST, and TaxiNet demonstrate over 30% fidelity improvement with minimal explanation cardinality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。