用动态分辨率加速扩散模型,提升图像修复速度与质量
Image Restoration via Diffusion Models with Dynamic Resolution

- 通过低维子空间投影降低计算复杂度
- SubDAPS++在多个任务中实现更快修复且保持高保真
- 适合需要高效图像修复的工程应用
扩散模型(DMs)在各类图像修复任务中表现出色。然而,现有方法通常在高维像素空间操作,导致计算开销大。基于潜在空间的扩散模型虽利用变分自编码器压缩表示以缓解问题,但需重复编码器-解码器推理,带来显著额外负担,运行效率甚至低于像素空间方法。为解决此效率瓶颈,本文提出使用动态分辨率扩散模型将数据投影至低维子空间,加速推理过程。我们首先微调预训练扩散模型以获得动态分辨率先验,并将两种主流像素空间方法DPS和DAPS改造为子空间框架下的SubDPS与SubDAPS。鉴于SubDAPS在推理速度与重建保真度上的优势,进一步提出增强版本SubDAPS++,以同时提升修复效率与质量。在多种图像数据集和修复任务上的实证评估表明,所提方法在多数场景下优于近期基于扩散模型的方法。代码已公开于https://github.com/StarNextDay/SubDAPS.git。
原文摘要 · Abstract (English)
Diffusion models (DMs) have exhibited remarkable efficacy in various image restoration tasks. However, existing approaches typically operate within the high-dimensional pixel space, resulting in high computational overhead. While methods based on latent DMs seek to alleviate this issue by utilizing the compressed latent space of a variational autoencoder, they require repeated encoder-decoder inference. This introduces significant additional computational burdens, often resulting in runtime performance that is even inferior to that of their pixel-space counterparts. To mitigate the computational inefficiency, this work proposes projecting data into lower-dimensional subspaces using dynamic resolution DMs to accelerate the inference process. We first fine-tune pre-trained DMs for dynamic resolution priors and adapt DPS and DAPS, which are two widely used pixel-space methods for general image restoration tasks, into the proposed framework, yielding methods we refer to as SubDPS and SubDAPS, respectively. Given the favorable inference speed and reconstruction fidelity of SubDAPS, we introduce an enhanced variant termed SubDAPS++ to further boost both reconstruction efficiency and quality. Empirical evaluations across diverse image datasets and various restoration tasks demonstrate that the proposed methods outperform recent DM-based approaches in the majority of experimental scenarios. The code is available at https://github.com/StarNextDay/SubDAPS.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。