arXiv:2502.01048cs.CVcs.AI2025-02被引 4

提升视觉大模型可解释性,让模型的决策过程像人一样清晰可懂。

Sparks of Explainability: Recent Advancements in Explaining Large Vision Models

  • 用算法稳定性与索博尔指数评估注意力图,加速计算并增强可靠性。
  • 发现现有方法只知'关注位置',不知'感知内容',理解仍不充分。
  • 提出自动提取概念并可视化的CRAFT/MACO框架,适合研究可解释AI者。

本论文探索提升计算机视觉中深度神经网络可解释性的先进方法,通过分析和建模其特征利用机制。首先,基于算法稳定性与索博尔指数(Sobol indices)提出新评估指标,并结合准蒙特卡洛序列显著降低计算开销。EVA方法首次通过验证扰动分析为归因提供形式化保证。实验表明,在复杂场景下这些方法仍难以提供充分理解,因它们仅揭示模型'关注何处'而未说明'感知为何'。为此提出两个假设:一是通过模仿人类解释并优化1-Lipschitz函数空间实现模型与人类推理对齐;二是采用概念可解释性方法。提出CRAFT方法以自动化提取模型使用概念并评估其重要性,配合MACO实现可视化。上述工作整合为统一框架,并在ResNet模型上对ImageNet 1000类进行交互式演示。

原文摘要 · Abstract (English)

This thesis explores advanced approaches to improve explainability in computer vision by analyzing and modeling the features exploited by deep neural networks. Initially, it evaluates attribution methods, notably saliency maps, by introducing a metric based on algorithmic stability and an approach utilizing Sobol indices, which, through quasi-Monte Carlo sequences, allows a significant reduction in computation time. In addition, the EVA method offers a first formulation of attribution with formal guarantees via verified perturbation analysis. Experimental results indicate that in complex scenarios these methods do not provide sufficient understanding, particularly because they identify only "where" the model focuses without clarifying "what" it perceives. Two hypotheses are therefore examined: aligning models with human reasoning -- through the introduction of a training routine that integrates the imitation of human explanations and optimization within the space of 1-Lipschitz functions -- and adopting a conceptual explainability approach. The CRAFT method is proposed to automate the extraction of the concepts used by the model and to assess their importance, complemented by MACO, which enables their visualization. These works converge towards a unified framework, illustrated by an interactive demonstration applied to the 1000 ImageNet classes in a ResNet model.

可解释性视觉模型概念提取注意力分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。