arXiv:2411.17687cs.CV2024-11CVPR被引 17

用扩散模型生成多样退化图像,提升图像修复模型泛化能力

GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration

  • 基于扩散模型构建退化条件生成器,可合成六类真实退化图像
  • 生成超55万张退化图像,融合现有数据形成75万样本的GenDS数据集
  • 训练模型在分布外测试中性能显著提升,适合图像修复研究者使用

近年来,基于深度学习的全功能图像修复(AIOR)模型取得了显著进展。然而,其实际应用受限于对训练分布外样本的泛化能力不足,主要源于现有数据集中退化类型和场景多样性不够,难以覆盖真实世界复杂情况。此外,获取雾霾、低光、雨滴等大范围真实成对退化数据往往耗时且不可行。本文提出GenDeg,一种退化与强度感知的条件扩散模型,可从干净图像合成高质量退化图像。利用GenDeg,我们生成了超过55万张涵盖雾霾、雨水、积雪、运动模糊、低光和雨滴六类退化的图像样本,并将其与现有数据集融合,构建了包含超过75万样本的GenDS数据集。实验表明,基于GenDS训练的修复模型在分布外测试中表现显著优于仅使用原有数据集训练的模型。同时,本文还对基于扩散模型合成退化数据对AIOR的影响进行了全面分析。

原文摘要 · Abstract (English)

Deep learning-based models for All-In-One Image Restoration (AIOR) have achieved significant advancements in recent years. However, their practical applicability is limited by poor generalization to samples outside the training distribution. This limitation arises primarily from insufficient diversity in degradation variations and scenes within existing datasets, resulting in inadequate representations of real-world scenarios. Additionally, capturing large-scale real-world paired data for degradations such as haze, low-light, and raindrops is often cumbersome and sometimes infeasible. In this paper, we leverage the generative capabilities of latent diffusion models to synthesize high-quality degraded images from their clean counterparts. Specifically, we introduce GenDeg, a degradation and intensity-aware conditional diffusion model capable of producing diverse degradation patterns on clean images. Using GenDeg, we synthesize over 550k samples across six degradation types: haze, rain, snow, motion blur, low-light, and raindrops. These generated samples are integrated with existing datasets to form the GenDS dataset, comprising over 750k samples. Our experiments reveal that image restoration models trained on the GenDS dataset exhibit significant improvements in out-of-distribution performance compared to those trained solely on existing datasets. Furthermore, we provide comprehensive analyses on implications of diffusion model-based synthetic degradations for AIOR.

图像修复扩散模型数据合成泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。