无需微调，一键解析视觉模型内部表示，让黑箱变透明。
Multi-Depth Concept Extraction for Post-Hoc Vision Encoder Explanation
- 从多层编码器激活中聚类特征，生成可解释的分割图
- 概念与输入语义高度对齐，支持下游任务应用
- 适用于CNN和ViT，适合研究模型内部机制
面向视觉模型的可解释AI方法旨在识别影响最终预测的输入区域，并将其关联到人类可理解的概念。本文聚焦于编码器本身，通过神经激活视觉解释（NAVE）方法，从冻结的视觉编码器中间输出中提取并可视化内部表征。NAVE是一种后验、无监督、架构无关（适用于CNN和ViT）的方法，通过聚类多个编码器深度的复合特征激活，生成具有可控粒度的可解释分割图，无需微调或结构修改。大量实验表明，NAVE提取的概念与输入语义高度一致，且可应用于下游任务。此外，我们利用NAVE分析训练策略和架构对编码器表征的影响，证明其作为模型检查工具的有效性。总体而言，NAVE为视觉模型的后验分析与透明性提升提供了有力工具。
原文摘要 · Abstract (English)
Explainable AI methods for vision models aim to identify the parts of the input that are important for the final prediction and subsequently relate these regions to human-understandable concepts. Here, we propose focusing solely on the encoder and relating its intermediate outputs to the input, instead. We introduce Neuro-Activated Vision Explanations (NAVE), a post-hoc, unsupervised, and architecture-agnostic (across CNNs and ViTs) method for extracting and visualizing internal representations from frozen vision model encoders. Specifically, NAVE clusters composite feature activations from multiple encoder depths to produce interpretable segmentation maps with controllable granularity, requiring no fine-tuning or architectural modifications. Through extensive experiments, we quantitatively demonstrate that NAVE's concepts align with input semantics and can be used in downstream tasks. We further demonstrate NAVE as an inspection tool by analyzing how training strategies and architectures affect encoder representations. Overall, our results establish NAVE as an effective tool for post-hoc model inspection and enhancing transparency in vision models. \texttt{https://github.com/Ahcene-B/NAVE}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。