提升可解释AI概念的可读性,让模型决策更易懂。
Towards Human-Understandable Multi-Dimensional Concept Discovery
- 用SAM识别概念,结合专用遮蔽技术降噪
- 在保持解释忠实性前提下显著提升可读性
- 适合需要可信、清晰解释的AI应用开发者
概念可解释AI(C-XAI)旨在克服传统显著性图的局限,将像素转化为跨数据集一致且人类可理解的概念。其中,完备性衡量概念集解释模型决策的能力。多维概念发现(MCD)通过分解CNN隐空间为独立可解释子空间,有效提升完备性。然而,其解释仍难被人类理解,影响实际应用。为此,本文提出人可理解的多维概念发现(HU-MCD):利用分割一切模型(SAM)进行概念识别,并引入针对CNN的输入遮蔽技术,降低传统方法引入的噪声。结合完备性关系,HU-MCD在保持解释忠实性的前提下,显著提升概念可理解性。实验(含真人测试)表明,该方法比现有C-XAI方法更精确可靠。代码已开源:https://github.com/grobruegge/hu-mcd。
原文摘要 · Abstract (English)
Concept-based eXplainable AI (C-XAI) aims to overcome the limitations of traditional saliency maps by converting pixels into human-understandable concepts that are consistent across an entire dataset. A crucial aspect of C-XAI is completeness, which measures how well a set of concepts explains a model's decisions. Among C-XAI methods, Multi-Dimensional Concept Discovery (MCD) effectively improves completeness by breaking down the CNN latent space into distinct and interpretable concept subspaces. However, MCD's explanations can be difficult for humans to understand, raising concerns about their practical utility. To address this, we propose Human-Understandable Multi-dimensional Concept Discovery (HU-MCD). HU-MCD uses the Segment Anything Model for concept identification and implements a CNN-specific input masking technique to reduce noise introduced by traditional masking methods. These changes to MCD, paired with the completeness relation, enable HU-MCD to enhance concept understandability while maintaining explanation faithfulness. Our experiments, including human subject studies, show that HU-MCD provides more precise and reliable explanations than existing C-XAI methods. The code is available at https://github.com/grobruegge/hu-mcd.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。