arXiv:2511.18163cs.CV2025-11

提出可解释的匹配式少样本分割方法,让模型决策过程透明化。

Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design

论文配图:Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design
图 1 · 摘自论文原文
  • 利用多层特征匹配得分生成像素级归因图
  • 在多个基准数据集上显著优于传统可解释方法
  • 适合需要理解模型行为的少样本学习研究者

少样本语义分割(FSS)模型在仅有少量标注样本的情况下能实现强性能,但其决策过程仍不透明。尽管可解释人工智能在常规计算机视觉任务中进展迅速,但在少样本场景下的可解释性仍基本未被探索,而这对于理解模型行为和指导数据稀缺条件下的支持集选择至关重要。本文首次提出一种针对匹配式FSS模型的专门解释方法,利用其固有结构特性。所提出的Affinity Explainer通过计算支持图像与查询图像在多层特征上的匹配分数,提取出对查询分割预测贡献最大的支持图像像素区域的归因图。我们拓展了标准可解释性评估指标至FSS领域,并提出新指标以更好衡量解释在少样本场景中的实际效用。在多个FSS基准数据集上的全面实验表明,Affinity Explainer显著优于适配的传统归因方法。定性分析显示,解释结果呈现结构化、连贯的注意力模式,与模型架构一致,且有助于有效诊断模型。本工作为可解释少样本分割研究奠定了基础,推动更可靠少样本分割系统的构建。代码已公开于 https://github.com/pasqualedem/AffinityExplainer。

原文摘要 · Abstract (English)

Few-Shot Semantic Segmentation (FSS) models achieve strong performance in segmenting novel classes with minimal labeled examples, yet their decision-making processes remain largely opaque. While explainable AI has advanced significantly in standard computer vision tasks, interpretability in FSS remains virtually unexplored despite its critical importance for understanding model behavior and guiding support set selection in data-scarce scenarios. This paper introduces the first dedicated method for interpreting matching-based FSS models by leveraging their inherent structural properties. Our Affinity Explainer approach extracts attribution maps that highlight which pixels in support images contribute most to query segmentation predictions, using matching scores computed between support and query features at multiple feature levels. We extend standard interpretability evaluation metrics to the FSS domain and propose additional metrics to better capture the practical utility of explanations in few-shot scenarios. Comprehensive experiments on FSS benchmark datasets, using different models, demonstrate that our Affinity Explainer significantly outperforms adapted standard attribution methods. Qualitative analysis reveals that our explanations provide structured, coherent attention patterns that align with model architectures and and enable effective model diagnosis. This work establishes the foundation for interpretable FSS research, enabling better model understanding and diagnostic for more reliable few-shot segmentation systems. The source code is publicly available at https://github.com/pasqualedem/AffinityExplainer.

少样本分割可解释AI特征匹配归因图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。