统一修复多种图像退化，用扩散模型实现可控恢复
UniCoRN: Latent Diffusion-based Unified Controllable Image Restoration Network across Multiple Degradations
- 用多头控制网络+专家混合策略，统一处理模糊、噪声等多重退化
- 在多退化数据集上表现超越现有方法，严重退化图像也能有效恢复
- 适合需要端到端修复复杂真实图像的科研与工业应用
图像修复对计算机视觉任务至关重要，但现有方法通常仅针对单一退化类型（如模糊、噪声或雾霾）设计，难以应对现实中多种退化同时出现的情况。本文提出UniCoRN，一种基于潜在扩散模型的统一可控图像修复框架，可同时处理多种退化。我们发现图像中的低级视觉线索能有效引导可控扩散模型进行真实场景修复，并设计了基于专家混合策略的多头控制网络。模型通过精心设计的课程学习方案训练，无需预设具体退化类型。此外，我们还构建了MetaRestore——一个包含多种退化和伪影的金属镜头成像基准数据集。在多个挑战性数据集（包括自建基准）上的大量实验表明，该方法显著提升性能，能稳健恢复严重退化的图像。
原文摘要 · Abstract (English)
Image restoration is essential for enhancing degraded images across computer vision tasks. However, most existing methods address only a single type of degradation (e.g., blur, noise, or haze) at a time, limiting their real-world applicability where multiple degradations often occur simultaneously. In this paper, we propose UniCoRN, a unified image restoration approach capable of handling multiple degradation types simultaneously using a multi-head diffusion model. Specifically, we uncover the potential of low-level visual cues extracted from images in guiding a controllable diffusion model for real-world image restoration and we design a multi-head control network adaptable via a mixture-of-experts strategy. We train our model without any prior assumption of specific degradations, through a smartly designed curriculum learning recipe. Additionally, we also introduce MetaRestore, a metalens imaging benchmark containing images with multiple degradations and artifacts. Extensive evaluations on several challenging datasets, including our benchmark, demonstrate that our method achieves significant performance gains and can robustly restore images with severe degradations. Project page: https://codejaeger.github.io/unicorn-gh
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。