提出零样本分步修复机制,让模型自动处理复杂图像退化组合。
Chain-of-Restoration: Multi-Task Image Restoration Models are Zero-Shot Step-by-Step Universal Image Restorers
- 基于退化基底分步修复,无需训练所有退化组合。
- 在未知复合退化下实现媲美顶尖方法的修复效果。
- 适合需要泛化多种退化的图像修复场景。
尽管以往图像修复(IR)方法多聚焦于单一退化,近期研究逐渐关注由多种独立退化组合而成的复合退化问题。然而,现有针对复合退化的修复方法需构建包含指数级退化组合的训练数据,带来巨大负担。为此,本文提出通用图像修复(UIR)新任务设定:无需对所有退化组合进行训练,仅需在一组退化基底上训练,即可零样本地逐步移除任意可由这些基底构成的退化。受大语言模型思维链(Chain-of-Thought)启发,本文提出链式修复(Chain-of-Restoration, CoR)机制,引导模型逐步移除一种退化基底,持续进行直至图像完全恢复。通过在预训练多任务模型中引入简单退化判别器,CoR 实现了每步仅移除一个退化基底的流程。大量实验表明,CoR 在去除复合退化方面显著提升性能,在多数情况下达到或超过在所有退化组合上训练的先进方法水平。
原文摘要 · Abstract (English)
Despite previous image restoration (IR) methods have often concentrated on isolated degradations, recent research has increasingly focused on addressing composite degradations involving a complex combination of multiple isolated degradations. However, current IR methods for composite degradations require building training data that contain an exponential number of possible degradation combinations, which brings in a significant burden. To alleviate this issue, this paper proposes a new task setting, i.e. Universal Image Restoration (UIR). Specifically, UIR doesn't require training on all the degradation combinations but only on a set of degradation bases and then removing any degradation that these bases can potentially compose in a zero-shot manner. Inspired by the Chain-of-Thought that prompts large language models (LLMs) to address problems step-by-step, we propose Chain-of-Restoration (CoR) mechanism, which instructs models to remove unknown composite degradations step-by-step. By integrating a simple Degradation Discriminator into pre-trained multi-task models, CoR facilitates the process where models remove one degradation basis per step, continuing this process until the image is fully restored from the unknown composite degradation. Extensive experiments show that CoR can significantly improve model performance in removing composite degradations, achieving comparable or better results than those state-of-the-art (SoTA) methods trained on all degradations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。