arXiv:2409.17778cs.CV2024-09NeurIPS被引 17

用低分辨率图启动扩散模型,5步完成超分,速度提升5-7倍。

Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs

  • 用领域迁移SDE从低分辨率图开始扩散,融合预训练模型能力
  • 仅需5次采样步骤,真实与合成数据上均达顶尖性能
  • 适合追求高效高质图像超分的开发者和研究者

基于扩散模型的图像超分辨率(SR)因强大的图像修复能力而备受关注。然而,现有扩散模型往往难以在效率与性能间取得平衡:要么忽略预训练模型潜力,限制生成能力;要么需数十次从前向噪声开始推理,影响效率。本文提出DoSSR,一种基于领域迁移的扩散超分模型,利用预训练扩散模型的生成能力,同时通过从低分辨率(LR)图像启动扩散过程显著提升效率。核心是可无缝集成现有扩散模型的领域迁移方程,不仅增强扩散先验使用效果,也提高推理效率。进一步将离散迁移过程转化为连续形式,称为DoS-SDEs,实现快速定制化求解器。实验表明,该方法在合成与真实世界数据集上均达到领先性能,仅需5步采样。相比以往基于扩散先验的方法,提速5至7倍,展现卓越效率。代码已开源:https://github.com/QinpengCui/DoSSR。

原文摘要 · Abstract (English)

Diffusion-based image super-resolution (SR) models have attracted substantial interest due to their powerful image restoration capabilities. However, prevailing diffusion models often struggle to strike an optimal balance between efficiency and performance. Typically, they either neglect to exploit the potential of existing extensive pretrained models, limiting their generative capacity, or they necessitate a dozens of forward passes starting from random noises, compromising inference efficiency. In this paper, we present DoSSR, a Domain Shift diffusion-based SR model that capitalizes on the generative powers of pretrained diffusion models while significantly enhancing efficiency by initiating the diffusion process with low-resolution (LR) images. At the core of our approach is a domain shift equation that integrates seamlessly with existing diffusion models. This integration not only improves the use of diffusion prior but also boosts inference efficiency. Moreover, we advance our method by transitioning the discrete shift process to a continuous formulation, termed as DoS-SDEs. This advancement leads to the fast and customized solvers that further enhance sampling efficiency. Empirical results demonstrate that our proposed method achieves state-of-the-art performance on synthetic and real-world datasets, while notably requiring only 5 sampling steps. Compared to previous diffusion prior based methods, our approach achieves a remarkable speedup of 5-7 times, demonstrating its superior efficiency. Code: https://github.com/QinpengCui/DoSSR.

图像超分扩散模型高效推理领域迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。