通过对比中间特征与反向传播得分，实现细粒度图像匹配的精准可视化解释。
Pairwise Matching of Intermediate Representations for Fine-grained Explainability
- 融合模型中间激活与梯度相关性，生成成对图像的局部解释
- 在35个公开re-ID数据集上优于现有基线方法
- 专家认可其可视化对动物/建筑识别任务有实用价值
细粒度类别间的图像差异通常细微且局部，现有可解释性方法往往过于泛化，难以提供有用、可读的解释。我们提出一种新方法（PAIR-X），结合中间模型激活与反向传播的相关性分数，生成细粒度、高度局部化的成对视觉解释。以动物和建筑重识别（re-ID）为主要案例研究，在35个公开re-ID数据集上定性评估表明，该方法显著优于多种可解释性基线。专家访谈显示，动物re-ID领域专家认为PAIR-X是深度模型可解释性的实质性改进，并认为其可视化可直接应用于实际工作。我们还提出一种新型定量评估指标，证明即使模型相似度相同，PAIR-X对正确匹配对的可视化也更合理。通过提升可解释性，帮助人类更好区分正确与错误匹配。代码已开源：https://github.com/pairx-explains/pairx。
原文摘要 · Abstract (English)
The differences between images belonging to fine-grained categories are often subtle and highly localized, and existing explainability techniques for deep learning models are often too diffuse to provide useful and interpretable explanations. We propose a new explainability method (PAIR-X) that leverages both intermediate model activations and backpropagated relevance scores to generate fine-grained, highly-localized pairwise visual explanations. We use animal and building re-identification (re-ID) as a primary case study of our method, and we demonstrate qualitatively improved results over a diverse set of explainability baselines on 35 public re-ID datasets. In interviews, animal re-ID experts found PAIR-X to be a meaningful improvement over existing baselines for deep model explainability, and suggested that its visualizations would be directly applicable to their work. We also propose a novel quantitative evaluation metric for our method, and demonstrate that PAIR-X visualizations appear more plausible for correct image matches than incorrect ones even when the model similarity score for the pairs is the same. By improving interpretability, PAIR-X enables humans to better distinguish correct and incorrect matches. Our code is available at: https://github.com/pairx-explains/pairx
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。