提出新方法提升ViT可解释性，更准且快14倍以上
Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers
- 根据注意力头重要性动态分配相关性，改进跳连路径建模
- 在ImageNet1K和BloodMNIST上达到顶尖可解释性准确率
- 适合需要高效、精准模型解释的视觉AI研究者
视觉变换器（ViTs）难以解释，因现有相关性传播与注意力流动方法未充分考虑关键结构特性，如注意力头重要性不均及残差连接。以往方法通常假设注意力头重要性均等，且将跳连视为恒等路径，导致相关性分配不准。为此，我们提出GradSkip，一种基于自适应头加权与跳连感知传播的新型相关性传播方法。GradSkip建模注意力头差异重要性，并动态分配注意力与残差路径间的相关性。在ImageNet1K和BloodMNIST上的实验表明，GradSkip实现了当前最优的忠实度，同时所需计算量比表现最佳的现有方法减少超过14倍GFLOPs。基于Transformer的分割任务评估也显示其定位更精确，与真实标注区域对齐更好。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) are difficult to interpret because current methods of relevance propagation and attention flow do not fully consider some key architectural features, such as the uneven importance of attention heads and residual connections. Prior approaches typically assume uniform importance across attention heads; furthermore, they model skip connections as identity paths, leading to inaccurate relevance attribution. To address these issues, we introduce GradSkip, a novel relevance propagation method for ViTs based on adaptive head weighting and skip-aware propagation. GradSkip models the different importance of the attention heads and dynamically distributes relevance between the attention and residual paths. Experiments on ImageNet1K and BloodMNIST demonstrate a state-of-the-art faithfulness of GradSkip while requiring over 14 times fewer GFLOPs than the best-performing existing approaches. Additional evaluations using transformer-based segmentation confirm improved localization and alignment with ground-truth regions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。