用局部显著图生成半全局解释，发现模型依赖的视觉概念
Aggregating Local Saliency Maps for Semi-Global Explainable Image Classification
- 将图像分段后，用显著图量化各语义片段的影响
- 能发现模型对背景、水印等无关特征的错误依赖
- 适合调试分类器和分析模型决策逻辑
深度学习在图像分类中占主导地位，但理解模型如何做出预测仍具挑战。现有研究多聚焦单个预测的局部解释，如显著图，可可视化特定像素对预测的影响。然而，逐一审查大量解释难以实现，而全局方法常过于简化，忽略重要局部行为。为此，我们提出段落归因表（SATs），用于将局部显著性解释归纳为（半）全局洞察。SATs通过图像分段（如查基豪犬的“眼睛”）并利用显著图量化其影响，揭示模型在不同实例中依赖的关键概念，并暴露虚假关联（如对背景或水印的依赖），即便在分布外测试性能变化不大时仍可识别。SATs适用于任何可生成显著图的分类器，只需提供命名的分割图。该方法弥合了过度简化的全局总结与过于细节的局部解释之间的差距，为分析和调试图像分类器提供了实用工具。
原文摘要 · Abstract (English)
Deep learning dominates image classification tasks, yet understanding how models arrive at predictions remains a challenge. Much research focuses on local explanations of individual predictions, such as saliency maps, which visualise the influence of specific pixels on a model's prediction. However, reviewing many of these explanations to identify recurring patterns is infeasible, while global methods often oversimplify and miss important local behaviours. To address this, we propose Segment Attribution Tables (SATs), a method for summarising local saliency explanations into (semi-)global insights. SATs take image segments (such as "eyes" in Chihuahuas) and leverage saliency maps to quantify their influence. These segments highlight concepts the model relies on across instances and reveal spurious correlations, such as reliance on backgrounds or watermarks, even when out-of-distribution test performance sees little change. SATs can explain any classifier for which a form of saliency map can be produced, using segmentation maps that provide named segments. SATs bridge the gap between oversimplified global summaries and overly detailed local explanations, offering a practical tool for analysing and debugging image classifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。