用扩散模型初始化修复模型,快且还原真实细节。
Harnessing Diffusion-Yielded Score Priors for Image Restoration
- 用预训练扩散模型初始化修复网络,再做对抗训练。
- 单次前向传播完成修复,速度远超扩散方法。
- 支持文本控制和纹理调节,适合需要精细控制的场景。
深度图像修复模型旨在学习从退化图像空间到自然图像空间的映射。然而,它们面临去噪、生成真实细节及像素级一致性等关键挑战。现有方法主要分为基于MSE、GAN和扩散模型三类,但难以在修复质量、保真度与速度间取得良好平衡。本文提出HYPIR方法:先用预训练扩散模型初始化修复模型,再通过对抗训练微调,无需扩散损失、迭代采样或额外适配器。理论证明,该初始化使初始模型接近自然图像分布,提升数值稳定性,避免模式崩溃,并显著加速对抗训练收敛。此外,HYPIR继承扩散模型的丰富用户控制能力,支持文本引导修复和可调纹理丰富度。仅需一次前向传播,其收敛与推理速度均优于扩散方法。大量实验表明,HYPIR超越现有最先进方法,实现高效高质量图像修复。
原文摘要 · Abstract (English)
Deep image restoration models aim to learn a mapping from degraded image space to natural image space. However, they face several critical challenges: removing degradation, generating realistic details, and ensuring pixel-level consistency. Over time, three major classes of methods have emerged, including MSE-based, GAN-based, and diffusion-based methods. However, they fail to achieve a good balance between restoration quality, fidelity, and speed. We propose a novel method, HYPIR, to address these challenges. Our solution pipeline is straightforward: it involves initializing the image restoration model with a pre-trained diffusion model and then fine-tuning it with adversarial training. This approach does not rely on diffusion loss, iterative sampling, or additional adapters. We theoretically demonstrate that initializing adversarial training from a pre-trained diffusion model positions the initial restoration model very close to the natural image distribution. Consequently, this initialization improves numerical stability, avoids mode collapse, and substantially accelerates the convergence of adversarial training. Moreover, HYPIR inherits the capabilities of diffusion models with rich user control, enabling text-guided restoration and adjustable texture richness. Requiring only a single forward pass, it achieves faster convergence and inference speed than diffusion-based methods. Extensive experiments show that HYPIR outperforms previous state-of-the-art methods, achieving efficient and high-quality image restoration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。