量化CLIP类模型可解释性，提出新工具辅助理解模型内部机制
Quantifying and Enabling the Interpretability of CLIP-like Models
- 用文本分割与上下文学习分解注意力头属性
- 大模型比小模型更易解释，且属性在头内一致、跨头解耦
- 开源工具支持五种可解释性分析，适合研究者调试模型
CLIP 是当前最流行的视觉-语言基础模型之一，广泛应用于各类任务。然而，其内部工作机制仍不清晰。为填补这一空白，本文对六种来自 OpenAI 与 OpenCLIP 的 CLIP 类模型展开可解释性量化研究，这些模型在规模、预训练数据类型和图像块大小上各不相同。研究采用 TEXTSPAN 算法结合上下文学习，将注意力头分解为具体语义属性，并引入新指标衡量头内属性一致性与头间属性解耦程度。结果表明，较大模型普遍具有更高可解释性。为进一步帮助用户理解模型内部运作，本文提出 CLIP-InterpreT 工具，支持五种分析：基于属性的最近邻搜索、每头话题分割、对比分割、图像/文本的每头最近邻检索。
原文摘要 · Abstract (English)
CLIP is one of the most popular foundational models and is heavily used for many vision-language tasks. However, little is known about the inner workings of CLIP. To bridge this gap we propose a study to quantify the interpretability in CLIP like models. We conduct this study on six different CLIP models from OpenAI and OpenCLIP which vary by size, type of pre-training data and patch size. Our approach begins with using the TEXTSPAN algorithm and in-context learning to break down individual attention heads into specific properties. We then evaluate how easily these heads can be interpreted using new metrics which measure property consistency within heads and property disentanglement across heads. Our findings reveal that larger CLIP models are generally more interpretable than their smaller counterparts. To further assist users in understanding the inner workings of CLIP models, we introduce CLIP-InterpreT, a tool designed for interpretability analysis. CLIP-InterpreT offers five types of analyses: property-based nearest neighbor search, per-head topic segmentation, contrastive segmentation, per-head nearest neighbors of an image, and per-head nearest neighbors of text.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。