arXiv:2410.04224cs.CV2024-10NeurIPS被引 17

用大模型判别器实现一步扩散超分辨率,速度快3倍以上

Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator

  • 设计大型扩散判别器,直接从任意时间步提取噪声特征
  • 一步生成效果媲美百步模型,推理速度提升3倍以上
  • 适合需要快速高清图像生成的实时应用

扩散模型在真实世界图像超分辨率(Real-ISR)中表现优异,但计算成本高。现有方法多通过知识蒸馏或变分分数蒸馏将多步扩散模型压缩为单步模型,受限于教师模型能力。为此,我们提出一种新型单步扩散模型D³SR,其采用大规模扩散判别器,可在潜在空间中从任意时间步蒸馏噪声特征,突破教师模型性能瓶颈。同时,引入边缘感知的DISTS损失(EA-DISTS),增强细节生成能力。实验表明,与需数十甚至上百步的传统扩散方法相比,D³SR在定量指标和定性效果上均达到相当或更优表现,且推理速度至少提升3倍,参数量减少至少30%。代码与模型将公开于https://github.com/JianzeLi-114/D3SR。

原文摘要 · Abstract (English)

Diffusion models have demonstrated excellent performance for real-world image super-resolution (Real-ISR), albeit at high computational costs. Most existing methods are trying to derive one-step diffusion models from multi-step counterparts through knowledge distillation (KD) or variational score distillation (VSD). However, these methods are limited by the capabilities of the teacher model, especially if the teacher model itself is not sufficiently strong. To tackle these issues, we propose a new One-Step \textbf{D}iffusion model with a larger-scale \textbf{D}iffusion \textbf{D}iscriminator for SR, called D$^3$SR. Our discriminator is able to distill noisy features from any time step of diffusion models in the latent space. In this way, our diffusion discriminator breaks through the potential limitations imposed by the presence of a teacher model. Additionally, we improve the perceptual loss with edge-aware DISTS (EA-DISTS) to enhance the model's ability to generate fine details. Our experiments demonstrate that, compared with previous diffusion-based methods requiring dozens or even hundreds of steps, our D$^3$SR attains comparable or even superior results in both quantitative metrics and qualitative evaluations. Moreover, compared with other methods, D$^3$SR achieves at least $3\times$ faster inference speed and reduces parameters by at least 30\%. We will release code and models at https://github.com/JianzeLi-114/D3SR.

超分辨率扩散模型快速生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。