arXiv:2411.07445cs.CV2024-11被引 135

自适应提示模型提升雨雾图像修复效果,统一处理多种天气退化。

All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model

  • 用CLIP引导生成三类潜在提示,捕捉退化类型、属性和图像描述
  • 在扩散模型中融合退化提示,提升对不同天气的感知能力
  • 适合需要统一修复多种天气退化的实际应用场景

现有的全场景天气退化图像修复方法在利用退化先验方面效率不足,导致在不同天气条件下的适应性表现不佳。为此,我们提出一种自适应退化感知自提示模型(ADSM),用于全场景天气退化图像修复。具体而言,该模型利用对比语言-图像预训练模型(CLIP)来训练所提出的潜在提示生成器(LPGs),生成三类潜在提示:表征退化类型、退化属性及图像标题。此外,将获得的退化感知提示融入扩散模型的时间嵌入中,以增强退化感知能力;同时,利用潜在标题提示通过交叉注意力机制指导反向采样过程,实现精准图像重建。为进一步加速扩散模型的反向采样并弥补频率感知局限,引入小波导向噪声估计网络(WNE-Net)。在八个公开数据集上的大量实验表明,该方法在任务特定和全场景应用中均表现出色。

原文摘要 · Abstract (English)

Existing approaches for all-in-one weather-degraded image restoration suffer from inefficiencies in leveraging degradation-aware priors, resulting in sub-optimal performance in adapting to different weather conditions. To this end, we develop an adaptive degradation-aware self-prompting model (ADSM) for all-in-one weather-degraded image restoration. Specifically, our model employs the contrastive language-image pre-training model (CLIP) to facilitate the training of our proposed latent prompt generators (LPGs), which represent three types of latent prompts to characterize the degradation type, degradation property and image caption. Moreover, we integrate the acquired degradation-aware prompts into the time embedding of diffusion model to improve degradation perception. Meanwhile, we employ the latent caption prompt to guide the reverse sampling process using the cross-attention mechanism, thereby guiding the accurate image reconstruction. Furthermore, to accelerate the reverse sampling procedure of diffusion model and address the limitations of frequency perception, we introduce a wavelet-oriented noise estimating network (WNE-Net). Extensive experiments conducted on eight publicly available datasets demonstrate the effectiveness of our proposed approach in both task-specific and all-in-one applications.

图像修复扩散模型天气退化自提示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。