通过抑制特征优化采样,提升视觉模型解释的可信度。
Sampling Matters in Explanations: Towards Trustworthy Attribution Analysis Building Block in Visual Models through Maximizing Explanation Certainty
- 用特征抑制法生成更贴近自然图像分布的样本
- 在ImageNet上显著提升解释确定性,优于现有方法
- 适合需要可靠模型解释的科研与工程场景
图像归因分析旨在突出视觉模型学习到的特征表示,使高亮特征图反映输入像素的重要性。梯度积分是归因分析的核心构建模块,通过整合多个衍生样本的梯度来突出与推理相关的语义特征。该模块常与其他信息(如激活图或注意力图)结合形成最终解释。然而,我们的理论分析表明,梯度积分中样本分布与自然图像分布的对齐程度决定了解释确定性的下界。先前工作在图像中添加噪声作为样本,但噪声分布导致解释确定性较低。反直觉的是,额外信息可能使神经网络饱和。因此,建立可信的归因分析需解决样本分布失配问题。我们提出一种半最优采样方法,通过抑制输入特征生成样本,其分布近似于自然图像分布。在大规模数据集ImageNet上的广泛定量评估表明,该方法有效,且在所有实验模型中均优于当前最优基线,生成更满意的解释。
原文摘要 · Abstract (English)
Image attribution analysis seeks to highlight the feature representations learned by visual models such that the highlighted feature maps can reflect the pixel-wise importance of inputs. Gradient integration is a building block in the attribution analysis by integrating the gradients from multiple derived samples to highlight the semantic features relevant to inferences. Such a building block often combines with other information from visual models such as activation or attention maps to form ultimate explanations. Yet, our theoretical analysis demonstrates that the extent to the alignment of the sample distribution in gradient integration with respect to natural image distribution gives a lower bound of explanation certainty. Prior works add noise into images as samples and the noise distributions can lead to low explanation certainty. Counter-intuitively, our experiment shows that extra information can saturate neural networks. To this end, building trustworthy attribution analysis needs to settle the sample distribution misalignment problem. Instead of adding extra information into input images, we present a semi-optimal sampling approach by suppressing features from inputs. The sample distribution by suppressing features is approximately identical to the distribution of natural images. Our extensive quantitative evaluation on large scale dataset ImageNet affirms that our approach is effective and able to yield more satisfactory explanations against state-of-the-art baselines throughout all experimental models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。