让AI模型在图像变形下仍能稳定识别关键证据,提升解释可信度。
Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations

- 用区域搜索提取紧凑且有区分性的视觉证据作为监督信号。
- 在ImageNet-100上使ViT模型归因稳定性显著提升,准确率仅降0.28点。
- 无需标注,适用于多种模型,适合关注可解释性与鲁棒性的研究者。
归因方法广泛用于刻画模型决策背后的证据,但其对模型行为的改进潜力尚未充分探索。在标签不变的几何变换下归因不一致,可能表明模型依赖于敏感于变换的证据,这促使我们提出归因正则化。然而,这种监督仅在归因能忠实反映决策依据时才有效。现有自监督方法通常对齐梯度类归因图(如Grad-CAM),但其忠实度有限,因此归因一致性并不等同于决策过程的一致性,导致变换鲁棒性问题未解。本文提出一种无需标注的归因正则化框架,基于图像区域的子模搜索。通过测量候选区域子集对模型输出的影响,搜索提取出紧凑、类别区分性强的证据作为搜索生成的监督信号。进一步引入一种包含路径一致性与终止对齐项的子模排序损失,分别对齐成对搜索轨迹中空间对应候选物的排名,并促使变换后轨迹在目标终止步满足停止条件。该损失为最终归因及原本离散的证据选择过程提供了可微分代理。在ImageNet-100上的实验表明,本方法显著提升了ViT-B/16的归因稳定性、插入/删除得分,准确率仅下降0.28点;在ViT-L/16上也获得类似提升。在ImageNet-1K上,它提升了ResNet-50和ConvNeXt-B在变换输入下的准确率,同时将干净输入准确率下降控制在0.30点以内,证明了更一致的证据依赖性与极小性能损耗。代码即将发布。
原文摘要 · Abstract (English)
Attribution methods are widely used to characterize the evidence underlying model predictions, yet their potential to improve model behavior remains underexplored. Attribution inconsistency under label-preserving geometric transformations may indicate transformation-sensitive evidence reliance, motivating attribution regularization. However, such supervision is valid only when attribution faithfully reflects the evidence driving predictions. Existing self-supervised methods typically align gradient-based maps such as Grad-CAM, whose limited faithfulness means that attribution consistency need not imply consistency of the underlying decision process, leaving transformation robustness unresolved. We propose an annotation-free attribution regularization framework based on submodular search over image regions. By measuring how candidate subsets affect model outputs, the search extracts compact, class-discriminative evidence as search-derived supervision. We further introduce a submodular ranking loss with path-consistency and termination-alignment terms that respectively align spatially corresponding candidate rankings along paired search trajectories and encourage the transformed trajectory to satisfy the stopping criterion at the target terminal step. The loss provides a differentiable surrogate for regularizing both final attributions and the otherwise discrete evidence-selection process. Experiments on ImageNet-100 show that our method substantially improves attribution stability, Insertion, and Deletion on ViT-B/16 with only a 0.28-point accuracy drop, with similar gains on ViT-L/16. On ImageNet-1K, it improves transformed-input accuracy on ResNet-50 and ConvNeXt-B while limiting the clean-accuracy drop to 0.30 points, demonstrating more consistent evidence reliance with minimal performance loss. Code will be released soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。