arXiv:2409.09497cs.CVcs.AI2024-09中稿 · WACV 2025被引 4

通过多尺度原型学习,让语义分割结果更可解释。

Multi-Scale Grouped Prototypes for Interpretable Semantic Segmentation

  • 引入多尺度原型层,学习不同尺度的典型图像块
  • 稀疏分组机制使模型更稀疏,提升可解释性
  • 在多个数据集上表现优于现有可解释方法

原型化部件学习正成为提升语义分割可解释性的有力方法。模型从训练中见过的真实图像块中选取原型,并基于测试图像部件与原型的相似性生成稠密预测图,从而让用户可追溯预测结果与模型学习模式之间的关联。本文提出一种基于多尺度图像表示的可解释语义分割方法。首先,设计原型层以在多个尺度显式学习多样化的原型部件,生成多尺度原型激活输出;其次,提出稀疏分组机制,将各尺度的原型部件聚为多尺度稀疏组,深化对多尺度物体表征间交互的理解,同时增强模型可解释性。在Pascal VOC、Cityscapes和ADE20K上的实验表明,该方法提升了模型稀疏性,改善了可解释性,并缩小了与非可解释模型的性能差距。代码已开源。

原文摘要 · Abstract (English)

Prototypical part learning is emerging as a promising approach for making semantic segmentation interpretable. The model selects real patches seen during training as prototypes and constructs the dense prediction map based on the similarity between parts of the test image and the prototypes. This improves interpretability since the user can inspect the link between the predicted output and the patterns learned by the model in terms of prototypical information. In this paper, we propose a method for interpretable semantic segmentation that leverages multi-scale image representation for prototypical part learning. First, we introduce a prototype layer that explicitly learns diverse prototypical parts at several scales, leading to multi-scale representations in the prototype activation output. Then, we propose a sparse grouping mechanism that produces multi-scale sparse groups of these scale-specific prototypical parts. This provides a deeper understanding of the interactions between multi-scale object representations while enhancing the interpretability of the segmentation model. The experiments conducted on Pascal VOC, Cityscapes, and ADE20K demonstrate that the proposed method increases model sparsity, improves interpretability over existing prototype-based methods, and narrows the performance gap with the non-interpretable counterpart models. Code is available at github.com/eceo-epfl/ScaleProtoSeg.

语义分割可解释性原型学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。