用轻量适配器实现高效盲图像修复，参数减少36倍
BIR-Adapter: A parameter-efficient diffusion adapter for blind image restoration
- 设计可即插即用的注意力机制，仅训练少量参数
- 在合成与真实退化数据上表现优于或媲美主流方法
- 适合希望低开销改造现有扩散模型的研究者
我们提出 BIR-Adapter，一种用于盲图像修复的参数高效扩散适配器。基于预训练扩散模型在图像退化下仍能保留有效表征的观察，BIR-Adapter 引入了一种参数高效的即插即用注意力机制，显著降低可训练参数数量。为提升恢复可靠性，还引入采样引导机制以缓解重建过程中的幻觉问题。在合成与真实世界退化数据上的实验表明，BIR-Adapter 在多个场景下达到或超越当前最优性能，同时所需训练参数最多减少36倍。此外，该适配器结构可轻松集成至现有模型中。我们通过将仅支持超分辨率的扩散模型扩展为可处理未知退化的通用修复模型，验证了该方法的通用性，展示了其在更广泛图像修复任务中的适应能力。
原文摘要 · Abstract (English)
We introduce the BIR-Adapter, a parameter-efficient diffusion adapter for blind image restoration. Diffusion-based restoration methods have demonstrated promising performance in addressing this fundamental problem in computer vision, typically relying on auxiliary feature extractors or extensive fine-tuning of pre-trained models. Building on the observation that large-scale pretrained diffusion models can retain informative representations under image degradations, BIR-Adapter introduces a parameter-efficient, plug-and-play attention mechanism that substantially reduces the number of trained parameters. To further improve reliability, we adapt a sampling guidance mechanism that mitigates hallucinations during restoration. Experiments on synthetic and real-world degradations demonstrate that BIR-Adapter achieves competitive, and in several settings superior, performance compared to state-of-the-art methods while requiring up to 36x fewer trained parameters. Moreover, the adapter-based design enables integration into existing models. We validate this generality by extending a super-resolution-only diffusion model to handle additional unknown degradations, highlighting the adaptability of our approach for broader image restoration tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。