用图文对比模型分析图像识别偏差，找关键决策依据。
Caption-Driven Explainability: Probing CNNs for Bias via CLIP
- 将待解释模型嵌入CLIP，通过文本描述定位关键判别概念
- 在多个数据集上验证，能准确识别模型依赖的主导概念
- 适合研究模型偏见、提升AI鲁棒性的研究人员使用
机器学习的鲁棒性已成为关键挑战。可解释人工智能（XAI）旨在理解模型行为并提升其鲁棒性。当前主流的计算机视觉XAI方法是生成显著图，但若存在虚假且显著的特征，该方法可能误导判断。本文提出一种基于图像描述的XAI方法，通过一种新型网络手术技术，将待解释的独立模型整合进对比语言-图像预训练（CLIP）模型中。新模型能够识别对预测贡献最大的主导概念，降低模型因协变量偏移而误判的风险，显著推动了鲁棒机器学习的发展。代码已公开于https://github.com/patch0816/caption-driven-xai。
原文摘要 · Abstract (English)
Robustness has become one of the most critical problems in machine learning (ML). The science of interpreting ML models to understand their behavior and improve their robustness is referred to as explainable artificial intelligence (XAI). One of the state-of-the-art XAI methods for computer vision problems is to generate saliency maps. A saliency map highlights the pixel space of an image that excites the ML model the most. However, this property could be misleading if spurious and salient features are present in overlapping pixel spaces. In this paper, we propose a caption-based XAI method, which integrates a standalone model to be explained into the contrastive language-image pre-training (CLIP) model using a novel network surgery approach. The resulting caption-based XAI model identifies the dominant concept that contributes the most to the models prediction. This explanation minimizes the risk of the standalone model falling for a covariate shift and contributes significantly towards developing robust ML models. Our code is available at https://github.com/patch0816/caption-driven-xai
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。