CLDM在图像修复中表现不佳，传统方法反而更优。
Are Conditional Latent Diffusion Models Effective for Image Restoration?
- 用低层特征建模退化与真实图像关系，传统方法优于扩散模型。
- 在轻微退化场景下，扩散模型失真严重，性能反不如传统方法。
- 揭示了条件扩散模型在图像修复中的局限性，适合反思现有方案。
近期图像修复领域越来越多采用条件潜空间扩散模型（CLDMs），尽管其在近年表现出显著性能提升，本文质疑其在图像修复任务中的适用性。CLDMs擅长捕捉高层语义关联，适用于带空间条件的文本生成任务。但在图像修复中，目标是提升视觉质量，而这类模型难以通过低层表示建模退化图像与真实图像之间的关系。通过大量实验，我们对比了最先进的CLDMs与传统修复模型，结果表明，尽管CLDMs具有规模优势，却普遍存在高失真和语义偏差问题，尤其在弱退化情况下，传统方法反而表现更佳。此外，我们还对CLDM设计要素的影响进行了实证分析。希望本研究能促使学界重新审视当前基于CLDM的修复方案，为该领域带来更多新可能。
原文摘要 · Abstract (English)
Recent advancements in image restoration increasingly employ conditional latent diffusion models (CLDMs). While these models have demonstrated notable performance improvements in recent years, this work questions their suitability for IR tasks. CLDMs excel in capturing high-level semantic correlations, making them effective for tasks like text-to-image generation with spatial conditioning. However, in IR, where the goal is to enhance image perceptual quality, these models face difficulty of modeling the relationship between degraded images and ground truth images using a low-level representation. To support our claims, we compare state-of-the-art CLDMs with traditional image restoration models through extensive experiments. Results reveal that despite the scaling advantages of CLDMs, they suffer from high distortion and semantic deviation, especially in cases with minimal degradation, where traditional methods outperform them. Additionally, we perform empirical studies to examine the impact of various CLDM design elements on their restoration performance. We hope this finding inspires a reexamination of current CLDM-based IR solutions, opening up more opportunities in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。