对比13种解释方法在多种视觉模型上的表现,发现解释效果不跨架构通用。
Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

- 构建标准化框架,评估13种方法在8类模型上的5项指标
- 基于注意力图的方法在卷积网络表现好,但在线性注意力模型差
- 单一指标评价易失真,需多维度综合判断解释质量
现有可解释人工智能(XAI)研究多基于卷积神经网络(CNN),对当前主流的视觉变压器(ViT)架构是否适用缺乏系统验证。本文提出一个受控基准测试,从忠实性、定位精度、鲁棒性、复杂度和计算成本五个维度评估13种来自四类算法家族的解释方法。实验覆盖八种代表性骨干网络:CNN、各向同性ViT、分层变压器、混合架构及线性注意力变压器。结果表明,解释性能高度依赖模型架构,传统在CNN上建立的排名无法可靠迁移到变压器模型。基于类激活图(CAM)的方法在多数模型上取得高定位分数,但其优势主要源于边界框评估的饱和效应而非真实定位能力;在全局注意力变压器上,这些方法鲁棒性差,而注意力传播法虽稳定但定位不佳。此外,忠实性相关性对不同方法区分度有限,凸显单指标评价的局限。研究挑战了现有结论,强调应采用架构感知的多维度评估。代码与结果已开源:https://github.com/Nishan-Charlie/VIT_XAI_Bench。
原文摘要 · Abstract (English)
Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural networks (CNNs), with limited investigation into whether these conclusions generalize to the diverse Vision Transformer (ViT) architectures that now dominate computer vision. This paper presents a controlled benchmark that evaluates attribution quality across five dimensions: faithfulness, localization, robustness, complexity, and computational cost. A standardized framework assesses 13 attribution methods from four algorithmic families on eight representative backbones spanning CNNs, isotropic ViTs, hierarchical transformers, hybrid architectures, and linear-attention transformers. The results show that attribution performance is strongly architecture-dependent and that rankings established on CNNs do not reliably transfer to transformer-based models. CAM-based methods achieve the highest scores under the conventional bounding-box localization metric on CNNs and most ViTs but perform poorly on linear-attention architectures. Pixel-level dense-mask evaluation further reveals that these gains largely reflect metric saturation rather than accurate localization. CAM-based methods also exhibit limited robustness on global-attention transformers, whereas attention rollout provides consistently stable explanations with poor localization. Furthermore, faithfulness correlation offers limited discrimination between attribution methods, highlighting the limitations of single-metric evaluation. These findings challenge prevailing conclusions on attribution performance and demonstrate the need for architecture-aware, multi-dimensional evaluation. The open-source code for the evaluation framework and benchmark results is available at https://github.com/Nishan-Charlie/VIT_XAI_Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。