用预训练GAN特征引导扩散模型，提升图像修复质量。
GAN-Diff : Coupling Pretrained WGAN-GP Features with Conditional Diffusion U-Nets
- 将冻结的WGAN-GP中间特征通过交叉注意力注入扩散U-Net
- 在去噪和超分辨率任务上分别提升4.40 dB和3.70 dB PSNR
- 适合需要高质量图像修复的研究者与工程应用
生成对抗网络（GAN）可高效生成图像，而扩散模型虽能实现高质量图像修复，但需迭代采样。本文提出一种混合框架，利用预训练的带梯度惩罚的水印生成对抗网络（WGAN-GP）作为条件扩散图像修复的特征先验。冻结的WGAN-GP生成器中间特征通过交叉注意力融入扩散U-Net，并在DDIM采样过程中保持固定。该框架在CelebA人脸图像上评估了高斯去噪与2X超分辨率两项修复任务。开发中识别并解决了多种不稳定性问题，包括对抗学习率失衡、扩散初始化不当、过量污染及参数平均不足。结果表明，该框架持续提升了退化与低分辨率图像的质量，在去噪任务上比基线提升4.40 dB PSNR，超分辨率任务上提升3.70 dB PSNR。这证明冻结的GAN特征先验可有效引导扩散模型实现稳定高效的图像修复。
原文摘要 · Abstract (English)
Generative adversarial networks (GANs) can provide efficient image generation, while diffusion models offer high-quality image restoration but require iterative sampling. This paper presents a hybrid GAN-guided diffusion framework that uses a pretrained Wasserstein GAN with gradient penalty (WGAN-GP) as a feature prior for conditional diffusion-based image restoration. Intermediate features from the frozen WGAN-GP generator are incorporated into a diffusion U-Net through cross-attention and remain fixed during the DDIM sampling process. The framework is evaluated on two restoration tasks, Gaussian denoising and 2Xsuper-resolution, using CelebA face images. During development, several sources of instability were identified and addressed, including adversarial learning-rate imbalance, inappropriate diffusion initialization, excessive corruption, and insufficient parameter averaging. The resulting framework consistently improves the quality of both degraded and low-resolution images. In particular, it improves denoising performance by 4.40 dB in PSNR and super-resolution performance by 3.70 dB over their respective input baselines. These results demonstrate the potential of a frozen GAN feature prior to guide diffusion models toward stable and effective image restoration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。