arXiv:2412.18604cs.CV2024-12被引 3

用文生图扩散模型解释分类器决策,支持多层级语义分析。

Explaining in Diffusion: Explaining a Classifier Through Hierarchical Semantics with Text-to-Image Diffusion Models

  • 基于文本到图像扩散模型构建分层语义解释框架
  • 可覆盖复杂场景中多个概念的细粒度语义(如胡须类型)
  • 无需为每个分类器重训练,适用于人脸、动物等场景

分类器是众多计算机视觉任务的核心组件,但理解其决策过程仍具挑战。本文提出DiffEx,利用文本到图像扩散模型的能力解释分类器决策。与传统依赖GAN的可解释方法不同,后者仅限于单一概念且需为每类分类器单独训练,而DiffEx能解释聚焦单一概念(如人脸或动物)及包含多重概念的复杂场景分类器。该方法通过视觉-语言模型生成分层语义列表,使用户不仅能识别影响分类的主要语义(如面部分类器中的'胡子'),还能定位子类别(如'山羊胡'或'巴尔博胡须')。实验表明,相比传统GAN方法,DiffEx能覆盖更广泛的语义层次,提供更细致的决策解释工具。

原文摘要 · Abstract (English)

Classifiers are important components in many computer vision tasks, serving as the foundational backbone of a wide variety of models employed across diverse applications. However, understanding the decision-making process of classifiers remains a significant challenge. We propose DiffEx, a novel method that leverages the capabilities of text-to-image diffusion models to explain classifier decisions. Unlike traditional GAN-based explainability models, which are limited to simple, single-concept analyses and typically require training a new model for each classifier, our approach can explain classifiers that focus on single concepts (such as faces or animals) as well as those that handle complex scenes involving multiple concepts. DiffEx employs vision-language models to create a hierarchical list of semantics, allowing users to identify not only the overarching semantic influences on classifiers (e.g., the 'beard' semantic in a facial classifier) but also their sub-types, such as 'goatee' or 'Balbo' beard. Our experiments demonstrate that DiffEx is able to cover a significantly broader spectrum of semantics compared to its GAN counterparts, providing a hierarchical tool that delivers a more detailed and fine-grained understanding of classifier decisions.

可解释性扩散模型语义分析分类器解释

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。