arXiv:2412.05134cs.CVcs.LG2024-12ECCV被引 1

用SE模块生成模型解释,不改架构也能看懂决策依据

How to Squeeze An Explanation Out of Your Model

  • 在分类层前加SE模块,通过调节注意力向量提取关键特征
  • 在人脸与行为生物识别数据集上均有效,且不影响原任务性能
  • 适用于图像和视频多模态场景,对模型类型无依赖

深度学习模型虽可靠,但缺乏决策解释,尤其在生物识别、安全与医疗等敏感领域成短板。现有可解释性方法多基于梯度反传生成视觉热图,但仅适配标准图像模型,难以拓展至视频或多模态场景及自定义架构。本文提出一种模型无关的可解释性方法,创新性地利用挤压-激励(Squeeze and Excitation, SE)块生成注意力热图。通过在任意模型的分类层前引入SE块,并操控其SE向量,可提取最具影响力的特征。实验表明,该方法在人脸特征(CelebA)和行为生物识别(Active Speaker Detection)数据集上均有效,且不损害原模型性能,在主流物体数据集上表现媲美现有先进方法,展现出对不同数据类型的强鲁棒性。

原文摘要 · Abstract (English)

Deep learning models are widely used nowadays for their reliability in performing various tasks. However, they do not typically provide the reasoning behind their decision, which is a significant drawback, particularly for more sensitive areas such as biometrics, security and healthcare. The most commonly used approaches to provide interpretability create visual attention heatmaps of regions of interest on an image based on models gradient backpropagation. Although this is a viable approach, current methods are targeted toward image settings and default/standard deep learning models, meaning that they require significant adaptations to work on video/multi-modal settings and custom architectures. This paper proposes an approach for interpretability that is model-agnostic, based on a novel use of the Squeeze and Excitation (SE) block that creates visual attention heatmaps. By including an SE block prior to the classification layer of any model, we are able to retrieve the most influential features via SE vector manipulation, one of the key components of the SE block. Our results show that this new SE-based interpretability can be applied to various models in image and video/multi-modal settings, namely biometrics of facial features with CelebA and behavioral biometrics using Active Speaker Detection datasets. Furthermore, our proposal does not compromise model performance toward the original task, and has competitive results with current interpretability approaches in state-of-the-art object datasets, highlighting its robustness to perform in varying data aside from the biometric context.

可解释性SE模块生物识别多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。