用扩散模型生成图像，让大模型解释视觉分类器决策逻辑。
DEXTER: Diffusion-Guided EXplanations with TExtual Reasoning for Vision Models
- 用扩散模型优化文本提示，生成类条件图像
- 在ImageNet等数据集上优于现有方法
- 无需训练数据即可揭示模型偏差和决策模式
理解与解释机器学习模型行为是构建透明可信AI系统的关键。我们提出DEXTER，一种无需训练数据的框架，利用扩散模型与大语言模型生成视觉分类器的全局性、文本化解释。DEXTER通过优化文本提示，合成能强烈激活目标分类器的类条件图像，并基于这些合成样本生成描述类别特异性决策模式与偏见的自然语言报告。与以往工作不同，DEXTER在无训练数据或真实标签的情况下，仍可生成关于分类器决策过程的自然语言解释。我们在激活最大化、切片发现与去偏、偏差解释三个任务中验证其灵活性，展示了其揭示视觉分类器内部机制的能力。定量与定性评估，包括用户研究，表明DEXTER输出准确且可解释。在ImageNet、Waterbirds、CelebA和FairFaces上的实验显示，DEXTER在全局模型解释与类别级偏差报告方面均优于现有方法。代码已开源。
原文摘要 · Abstract (English)
Understanding and explaining the behavior of machine learning models is essential for building transparent and trustworthy AI systems. We introduce DEXTER, a data-free framework that employs diffusion models and large language models to generate global, textual explanations of visual classifiers. DEXTER operates by optimizing text prompts to synthesize class-conditional images that strongly activate a target classifier. These synthetic samples are then used to elicit detailed natural language reports that describe class-specific decision patterns and biases. Unlike prior work, DEXTER enables natural language explanation about a classifier's decision process without access to training data or ground-truth labels. We demonstrate DEXTER's flexibility across three tasks-activation maximization, slice discovery and debiasing, and bias explanation-each illustrating its ability to uncover the internal mechanisms of visual classifiers. Quantitative and qualitative evaluations, including a user study, show that DEXTER produces accurate, interpretable outputs. Experiments on ImageNet, Waterbirds, CelebA, and FairFaces confirm that DEXTER outperforms existing approaches in global model explanation and class-level bias reporting. Code is available at https://github.com/perceivelab/dexter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。