arXiv:2412.14630cs.CV2024-12被引 5

用扩散模型统一图像修复与增强,通过频域校准提升质量。

Unified Image Restoration and Enhancement: Degradation Calibrated Cycle Reconstruction Diffusion Model

  • 两阶段扩散推理学习退化、粗糙正常与正常图像映射关系。
  • 在小波低频域进行精细化校准,高频频域用特征增益模块去冗余。
  • 支持多任务通用,少量训练样本即达顶尖重建与感知质量。

图像修复与增强对众多计算机视觉应用至关重要,但高效统一这两项任务仍具挑战。受扩散模型迭代精炼能力启发,本文提出CycleRDM框架,实现修复与增强的统一,并达成高质量映射。具体地,CycleRDM通过两阶段扩散推理学习退化域、粗糙正常域与正常域之间的映射关系;随后利用离散小波变换将最终校准过程转移至小波低频域,从频域视角借助任务特定频带空间进行细粒度校准。为提升修复质量,设计了针对分解后小波高频域的特征增益模块以消除冗余特征;同时引入多模态文本提示与傅里叶变换,驱动稳定去噪并降低推断过程中的随机性。经广泛验证,CycleRDM可有效泛化至多种图像修复与增强任务,仅需少量训练样本即在多个重建与感知质量基准上表现显著优越。源码将发布于 https://github.com/hejh8/CycleRDM。

原文摘要 · Abstract (English)

Image restoration and enhancement are pivotal for numerous computer vision applications, yet unifying these tasks efficiently remains a significant challenge. Inspired by the iterative refinement capabilities of diffusion models, we propose CycleRDM, a novel framework designed to unify restoration and enhancement tasks while achieving high-quality mapping. Specifically, CycleRDM first learns the mapping relationships among the degraded domain, the rough normal domain, and the normal domain through a two-stage diffusion inference process. Subsequently, we transfer the final calibration process to the wavelet low-frequency domain using discrete wavelet transform, performing fine-grained calibration from a frequency domain perspective by leveraging task-specific frequency spaces. To improve restoration quality, we design a feature gain module for the decomposed wavelet high-frequency domain to eliminate redundant features. Additionally, we employ multimodal textual prompts and Fourier transform to drive stable denoising and reduce randomness during the inference process. After extensive validation, CycleRDM can be effectively generalized to a wide range of image restoration and enhancement tasks while requiring only a small number of training samples to be significantly superior on various benchmarks of reconstruction quality and perceptual quality. The source code will be available at https://github.com/hejh8/CycleRDM.

图像修复扩散模型频域校准小波变换

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。