让普通图像修复网络学会扩散模型的训练方式,提升修复泛化能力。
Elucidating and Endowing the Diffusion Training Paradigm for General Image Restoration
- 通过分析时间步、噪声层级等关系,重构扩散训练范式。
- 单任务修复泛化能力显著提升,多任务统一修复性能更优。
- 可无缝接入现有修复架构,适合追求通用性的研究者。
尽管扩散模型在图像修复(IR)任务中表现出强大的生成能力,但其复杂结构与迭代过程使其相比主流重建类通用修复网络难以实用。现有方法主要优化网络结构与扩散路径,却忽视了将扩散训练范式融入通用修复框架。本文通过系统分析时间步依赖、网络层次、噪声水平关系及多修复任务关联性,阐明了扩散训练范式适配通用修复训练的关键原则,提出一种基于扩散训练的新修复框架。为使修复网络同时具备修复能力与生成建模能力,引入一系列正则化策略,对齐扩散目标与修复任务,提升单任务下的泛化性。此外,针对不同修复任务受扩散生成影响程度差异,设计增量训练范式与任务特定适配器,进一步提升多任务统一修复性能。实验表明,该方法显著提升单任务修复的泛化能力,并在多任务统一修复中达到更优表现。值得注意的是,所提框架可无缝集成至现有通用修复架构中。
原文摘要 · Abstract (English)
While diffusion models demonstrate strong generative capabilities in image restoration (IR) tasks, their complex architectures and iterative processes limit their practical application compared to mainstream reconstruction-based general ordinary IR networks. Existing approaches primarily focus on optimizing network architecture and diffusion paths but overlook the integration of the diffusion training paradigm within general ordinary IR frameworks. To address these challenges, this paper elucidates key principles for adapting the diffusion training paradigm to general IR training through systematic analysis of time-step dependencies, network hierarchies, noise-level relationships, and multi-restoration task correlations, proposing a new IR framework supported by diffusion-based training. To enable IR networks to simultaneously restore images and model generative representations, we introduce a series of regularization strategies that align diffusion objectives with IR tasks, improving generalization in single-task scenarios. Furthermore, recognizing that diffusion-based generation exerts varying influences across different IR tasks, we develop an incremental training paradigm and task-specific adaptors, further enhancing performance in multi-task unified IR. Experiments demonstrate that our method significantly improves the generalization of IR networks in single-task IR and achieves superior performance in multi-task unified IR. Notably, the proposed framework can be seamlessly integrated into existing general IR architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。