用多低秩适配实现通用图像修复，提升质量与泛化能力
UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation
- 基于预训练生成模型，通过多低秩适配器实现跨退化类型修复
- 在多种混合退化场景下，重建图像在保真度和感知质量上均更优
- 适配器按退化相似性自动组合，适合处理复杂混合退化任务
现有统一方法通常将多退化图像修复视为多任务学习问题。尽管相比单退化修复方法表现更优，但忽略了多任务修复中的共性与特性，限制了模型性能。受深度生成模型和微调技术启发，我们提出一种基于多领域迁移学习中多低秩适配器（LoRA）的通用图像修复框架。该框架以预训练生成模型作为多退化修复的共享组件，并通过低秩适配将其迁移到特定退化任务。此外，提出一种基于退化相似性的LoRA组合策略，可自适应融合已训练的LoRA，使模型适用于混合退化修复。在多种及混合退化场景下的大量实验表明，所提方法不仅在重建图像保真度和感知质量上优于其他统一修复模型，且具备更强泛化能力。代码已开源：https://github.com/Justones/UIR-LoRA。
原文摘要 · Abstract (English)
Existing unified methods typically treat multi-degradation image restoration as a multi-task learning problem. Despite performing effectively compared to single degradation restoration methods, they overlook the utilization of commonalities and specificities within multi-task restoration, thereby impeding the model's performance. Inspired by the success of deep generative models and fine-tuning techniques, we proposed a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks using low-rank adaptation. Additionally, we introduce a LoRA composing strategy based on the degradation similarity, which adaptively combines trained LoRAs and enables our model to be applicable for mixed degradation restoration. Extensive experiments on multiple and mixed degradations demonstrate that the proposed universal image restoration method not only achieves higher fidelity and perceptual image quality but also has better generalization ability than other unified image restoration models. Our code is available at https://github.com/Justones/UIR-LoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。