arXiv:2501.19271cs.AIcs.LG2025-01被引 7

提出三类新指标,评估概念解释模型中概念是否存在及位置是否准确。

Concept-Based Explainable Artificial Intelligence: Metrics and Benchmarks

  • 设计概念全局重要性、存在性与位置三类量化指标
  • 发现多数关键概念实际并不存在于输入图像中
  • 揭示概念相关性导致激活区域错位,提醒慎用空间解释

基于概念的解释方法(如概念瓶颈模型,CBMs)通过将模型决策与人类可理解的概念关联,提升可解释性,其前提是这些概念能被准确映射到网络特征空间。然而该假设尚未经过严格验证,因领域缺乏标准化的评估指标与基准。为此,本文提出三种新指标:概念全局重要性、概念存在性与概念位置,并引入概念激活映射技术用于可视化。通过对后处理式CBMs的基准测试发现,许多被识别为最重要的概念实际上并不存在于输入图像中;即使存在,其显著性图也常错误地覆盖整个物体或误判特定区域。我们分析了概念自然相关性等根本原因,强调在需要精确空间解释的场景中,应更谨慎地应用此类技术。

原文摘要 · Abstract (English)

Concept-based explanation methods, such as concept bottleneck models (CBMs), aim to improve the interpretability of machine learning models by linking their decisions to human-understandable concepts, under the critical assumption that such concepts can be accurately attributed to the network's feature space. However, this foundational assumption has not been rigorously validated, mainly because the field lacks standardised metrics and benchmarks to assess the existence and spatial alignment of such concepts. To address this, we propose three metrics: the concept global importance metric, the concept existence metric, and the concept location metric, including a technique for visualising concept activations, i.e., concept activation mapping. We benchmark post-hoc CBMs to illustrate their capabilities and challenges. Through qualitative and quantitative experiments, we demonstrate that, in many cases, even the most important concepts determined by post-hoc CBMs are not present in input images; moreover, when they are present, their saliency maps fail to align with the expected regions by either activating across an entire object or misidentifying relevant concept-specific regions. We analyse the root causes of these limitations, such as the natural correlation of concepts. Our findings underscore the need for more careful application of concept-based explanation techniques especially in settings where spatial interpretability is critical.

可解释AI概念解释模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。