用小模块融合现有模型,实现高效高质图像重建。
A Modular Conditional Diffusion Framework for Image Reconstruction
- 模块化设计,仅需训练0.7M参数的小模块适配新任务。
- 采样效率提升4倍,性能不降,可与DDIM等加速技术兼容。
- 适合资源有限但需高质量图像重建的研究者和应用者。
扩散概率模型(DPMs)在盲图像恢复(IR)任务中表现出优异的感知质量,但现有方法存在任务特异性高、训练计算成本大等问题,限制了其在不同任务中的实用性和普及性,尤其对缺乏高性能算力和大量数据的用户而言。本文提出一种模块化扩散概率图像重建框架(DP-IR),可结合预训练的先进IR网络与生成式DPM,仅需额外训练一个0.7M参数的小模块即可适配特定IR任务。该框架支持一种采样策略,使神经函数评估次数至少减少4倍,且无性能损失,还可与DDIM等加速技术结合。我们在四组基准上评估了该方法在多帧超分辨率(burst JDD-SR)、动态场景去模糊和超分辨率任务上的表现,结果表明其在感知质量上优于现有方法,同时在保真度指标上保持竞争力。
原文摘要 · Abstract (English)
Diffusion Probabilistic Models (DPMs) have been recently utilized to deal with various blind image restoration (IR) tasks, where they have demonstrated outstanding performance in terms of perceptual quality. However, the task-specific nature of existing solutions and the excessive computational costs related to their training, make such models impractical and challenging to use for different IR tasks than those that were initially trained for. This hinders their wider adoption, especially by those who lack access to powerful computational resources and vast amount of training data. In this work we aim to address the above issues and enable the successful adoption of DPMs in practical IR-related applications. Towards this goal, we propose a modular diffusion probabilistic IR framework (DP-IR), which allows us to combine the performance benefits of existing pre-trained state-of-the-art IR networks and generative DPMs, while it requires only the additional training of a relatively small module (0.7M params) related to the particular IR task of interest. Moreover, the architecture of the proposed framework allows for a sampling strategy that leads to at least four times reduction of neural function evaluations without suffering any performance loss, while it can also be combined with existing acceleration techniques such as DDIM. We evaluate our model on four benchmarks for the tasks of burst JDD-SR, dynamic scene deblurring, and super-resolution. Our method outperforms existing approaches in terms of perceptual quality while it retains a competitive performance with respect to fidelity metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。