统一修复复杂图像退化,端到端搞定真实场景多种干扰
UniRes: Universal Image Restoration for Complex Degradations
- 用扩散模型融合多个专用修复模型,端到端处理混合退化
- 在复杂退化数据集上显著优于现有方法,单退化任务也表现稳定
- 只需单一退化训练数据,可灵活扩展,适合实际应用
真实世界图像修复面临多种退化,源于不同的拍摄条件、设备和后期处理流程。现有方法通过模拟退化并利用图像生成先验进行改进,但对真实场景数据的泛化能力仍不足。本文聚焦复杂退化——即多种已知退化类型的任意混合,这在真实场景中常见。提出一种简单而灵活的基于扩散的统一框架UniRes,可在扩散采样阶段组合多个专用模型,将多个隔离修复任务的知识迁移至复杂退化图像的修复中。该框架仅需针对各类退化类型收集隔离训练数据,具有高度灵活性,可通过统一公式扩展,并引入新范式调节保真度与质量权衡。在复杂退化及单退化图像修复数据集上均进行了评估,定性和定量实验表明,尤其在复杂退化图像上性能显著提升。
原文摘要 · Abstract (English)
Real-world image restoration is hampered by diverse degradations stemming from varying capture conditions, capture devices and post-processing pipelines. Existing works make improvements through simulating those degradations and leveraging image generative priors, however generalization to in-the-wild data remains an unresolved problem. In this paper, we focus on complex degradations, i.e., arbitrary mixtures of multiple types of known degradations, which is frequently seen in the wild. A simple yet flexible diffusionbased framework, named UniRes, is proposed to address such degradations in an end-to-end manner. It combines several specialized models during the diffusion sampling steps, hence transferring the knowledge from several well-isolated restoration tasks to the restoration of complex in-the-wild degradations. This only requires well-isolated training data for several degradation types. The framework is flexible as extensions can be added through a unified formulation, and the fidelity-quality trade-off can be adjusted through a new paradigm. Our proposed method is evaluated on both complex-degradation and single-degradation image restoration datasets. Extensive qualitative and quantitative experimental results show consistent performance gain especially for images with complex degradations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。