用生成参考动态调整修复过程，让图像恢复更准更灵活。
InstantIR: Blind Image Restoration with Instant Generative Reference
- 用预训练视觉编码器提取输入特征，实时生成修复参考
- 在多种退化程度下均达顶尖效果，视觉质量优秀
- 支持文本控制，可实现极端退化修复与创意修复
盲图像修复（BIR）面临测试时未知退化问题，需依赖强泛化能力。本文提出基于扩散模型的InstantIR方法，在推理阶段动态调整生成条件。首先通过预训练视觉编码器提取输入图像紧凑表征；在每步生成中，该表征被用于解码当前扩散隐变量，并实例化为生成先验。将退化图像编码此参考，提供鲁棒生成条件。我们观察到生成参考的方差随退化强度波动，据此设计自适应采样算法以匹配输入质量。大量实验表明，InstantIR达到当前最优性能，且视觉质量出色。通过调节生成参考的文本描述，可实现极端退化修复及创造性修复。
原文摘要 · Abstract (English)
Handling test-time unknown degradation is the major challenge in Blind Image Restoration (BIR), necessitating high model generalization. An effective strategy is to incorporate prior knowledge, either from human input or generative model. In this paper, we introduce Instant-reference Image Restoration (InstantIR), a novel diffusion-based BIR method which dynamically adjusts generation condition during inference. We first extract a compact representation of the input via a pre-trained vision encoder. At each generation step, this representation is used to decode current diffusion latent and instantiate it in the generative prior. The degraded image is then encoded with this reference, providing robust generation condition. We observe the variance of generative references fluctuate with degradation intensity, which we further leverage as an indicator for developing a sampling algorithm adaptive to input quality. Extensive experiments demonstrate InstantIR achieves state-of-the-art performance and offering outstanding visual quality. Through modulating generative references with textual description, InstantIR can restore extreme degradation and additionally feature creative restoration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。