利用预训练扩散模型的语义潜空间,实现高效无重训练的图像去雾。
Unleashing the Potential of the Semantic Latent Space in Diffusion Models for Image Dehazing
- 基于冻结预训练模型的语义潜空间,捕捉雾霾特征与时间步变化关系。
- 无需重训练和迭代采样,在多个数据集上超越现有去雾方法。
- 适合追求高效、低延迟去雾应用的研究者与开发者。
扩散模型因其强大的数据分布建模能力,被用于图像去雾任务。然而,其重训练成本高及推理时需大量采样步骤,限制了实际应用。本文探索冻结预训练扩散模型在语义潜空间中对雾霾图像的表征能力,提出一种基于扩散潜表示的去雾网络DiffLI$^2$D。研究发现,随着扩散时间步的变化,该潜空间可有效表达图像内容与雾霾特征。基于此,将不同时间步的潜表示融合至精心设计的去雾网络中,为去雾提供指导。所提方法避免了扩散模型的重训练与迭代采样过程,充分利用预训练模型的信息表示,为扩散模型引入去雾任务提供了新视角。在多个数据集上的实验表明,该方法性能优于现有去雾方法。代码已开源:https://github.com/aaaasan111/difflid。
原文摘要 · Abstract (English)
Diffusion models have recently been investigated as powerful generative solvers for image dehazing, owing to their remarkable capability to model the data distribution. However, the massive computational burden imposed by the retraining of diffusion models, coupled with the extensive sampling steps during the inference, limit the broader application of diffusion models in image dehazing. To address these issues, we explore the properties of hazy images in the semantic latent space of frozen pre-trained diffusion models, and propose a Diffusion Latent Inspired network for Image Dehazing, dubbed DiffLI$^2$D. Specifically, we first reveal that the semantic latent space of pre-trained diffusion models can represent the content and haze characteristics of hazy images, as the diffusion time-step changes. Building upon this insight, we integrate the diffusion latent representations at different time-steps into a delicately designed dehazing network to provide instructions for image dehazing. Our DiffLI$^2$D avoids re-training diffusion models and iterative sampling process by effectively utilizing the informative representations derived from the pre-trained diffusion models, which also offers a novel perspective for introducing diffusion models to image dehazing. Extensive experiments on multiple datasets demonstrate that the proposed method achieves superior performance to existing image dehazing methods. Code is available at https://github.com/aaaasan111/difflid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。