arXiv:2607.22544cs.AIcs.CV2026-07中稿 · the 4th World Conf…

用概念层控制生成,让图像修改更可控、可解释。

Concept-based Visual Counterfactual Explanations with Diffusion Models

论文配图:Concept-based Visual Counterfactual Explanations with Diffusion Models
图 1 · 摘自论文原文
  • 将分类器嵌入扩散模型,通过可解释概念引导修改
  • 在CelebA上实现更高翻转率且图像更贴近原图
  • 适合需要可信‘假设性’视觉解释的医疗等场景

视觉反事实解释旨在回答‘对这张图像做最小改动,如何使模型预测结果翻转’,在医疗等安全关键领域日益重要。现有基于扩散的方法虽能生成真实图像修改,但依赖外部分类器在噪声图像上可靠运行,导致脆弱且难部署。本文提出C-VCE,通过概念瓶颈层将分类器直接融入生成模型,使反事实生成由人类可理解的语义概念驱动,而非依赖对像素级扰动敏感的独立分类器。用户可在采样时开关特定概念,模型仅微调相关区域并保留其余部分,同时维持特征间相关性。为保证修改小而可控,引入概率正则化平衡‘改变预测’与‘贴近原图’,并使用梯度掩码限制修改范围至最相关区域。在CelebA等基准上,C-VCE达到或超越基线的翻转率,生成的反事实图像更接近输入且畸变更少。该方法使生成模型在需提供可信‘若…则…’示例的场景中更具实用性,也表明暴露和控制内部概念层是提升生成模型可解释性与安全性的重要方向。

原文摘要 · Abstract (English)

Visual counterfactual explanations aim to answer "what minimal change to this image would flip the model's prediction?", and are increasingly important as vision models are deployed in safety-critical domains (e.g., medicine). Existing diffusion-based methods can produce realistic edits, but they rely on external classifiers that must work reliably on noisy images, which makes them fragile and hard to deploy for robust explanations. We introduce C-VCE, a new diffusion framework that builds the classifier directly into the generative model via a concept bottleneck layer, so that counterfactuals are guided by human-interpretable features (concepts) instead of a separate noise robust classifier that works with pixel-level edits. Our model lets users to toggle on/off semantic concepts during sampling, then minimally adjusts relevant image regions, while preserving the rest of the image, respecting feature correlations. To keep edits small and controlled, we add a simple probabilistic regularizer that balances "change the prediction" against "stay close to the original", plus a gradient-based mask that confines modifications to the most relevant regions. On benchmarks such as CelebA, C-VCE matches or improves flip rates while producing counterfactuals that are visually closer to the input and less distorted than baselines that depend on separate noisy-image classifiers. These properties make C-VCE a practical tool for vision systems where users need concrete "what-if" images without having to trust an additional, noise-robust classifier. More broadly, our results suggest that exposing and controlling an internal concept layer is a promising way to make powerful generative models easier to understand and safer to use.

反事实解释扩散模型可解释AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。