用可解释AI识别扩散模型生成图像中的缺陷并优化修复
Refining Visual Artifacts in Diffusion Models via Explainable AI-based Flaw Activation Maps
- 通过可解释AI生成缺陷激活图，定位图像瑕疵区域
- 在正向和反向过程中分别增强缺陷区域噪声与关注度，提升重建质量
- 适用于图文生成、修复等任务，提升27.3%的生成质量
扩散模型在图像合成中取得显著成功，但消除伪影和不真实区域仍是关键挑战。本文提出自精炼扩散框架，通过基于可解释人工智能（XAI）的缺陷高亮器生成缺陷激活图（FAMs），精准识别伪影和不真实区域。FAMs在前向过程中放大缺陷区域的噪声，在反向过程中聚焦于这些区域，从而提升重建质量。该方法在多种扩散模型上实现高达27.3%的弗雷切特起始距离（FID）改进，在多个数据集和任务（包括图像生成、文本到图像生成、图像修复）中均表现稳健。结果表明，可解释AI不仅能用于解释，还能主动参与图像优化。该框架通用性强，显著推动图像合成领域发展。
原文摘要 · Abstract (English)
Diffusion models have achieved remarkable success in image synthesis. However, addressing artifacts and unrealistic regions remains a critical challenge. We propose self-refining diffusion, a novel framework that enhances image generation quality by detecting these flaws. The framework employs an explainable artificial intelligence (XAI)-based flaw highlighter to produce flaw activation maps (FAMs) that identify artifacts and unrealistic regions. These FAMs improve reconstruction quality by amplifying noise in flawed regions during the forward process and by focusing on these regions during the reverse process. The proposed approach achieves up to a 27.3% improvement in Fréchet inception distance across various diffusion-based models, demonstrating consistently strong performance on diverse datasets. It also shows robust effectiveness across different tasks, including image generation, text-to-image generation, and inpainting. These results demonstrate that explainable AI techniques can extend beyond interpretability to actively contribute to image refinement. The proposed framework offers a versatile and effective approach applicable to various diffusion models and tasks, significantly advancing the field of image synthesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。