用轻量适配器让图像修复网络更省参数、更高效。
Deep LoRA-Unfolding Networks for Image Restoration
- 共享预训练去噪器,每阶段用LoRA小模块动态调参。
- 相比传统方法,参数量最多减少N倍,性能相当或更好。
- 适合资源受限场景,如移动端部署或大规模图像修复。
深度展开网络(DUNs)将迭代优化与深度神经网络结合,在光谱成像重建、压缩感知和超分辨率等图像修复任务中表现优异。其结构由多个级联模块组成,每个模块包含梯度下降模块(GDM)和近端映射模块(PMM),后者在贝叶斯视角下等价于一个对已知噪声水平的高斯噪声进行去噪的去噪器。然而,现有DUNs存在两大缺陷:一是各阶段的PMM采用相同架构和去噪目标,无法适应不同阶段变化的噪声水平;二是结构重复导致参数冗余严重,内存开销大,难以部署于大规模或资源受限场景。为此,本文提出广义深度低秩适配(LoRA)展开网络(LoRun),通过共享单个预训练基础去噪器,并在各阶段的PMM中注入轻量级、阶段特异的LoRA适配器,动态调节去噪强度以匹配当前阶段噪声水平。该设计将核心恢复能力与任务适应解耦,无需复制完整网络参数即可实现精确控制,使N阶段的DUN最多可实现N倍参数压缩,同时保持相当或更优的性能。在三个图像修复任务上的大量实验验证了方法的高效性。
原文摘要 · Abstract (English)
Deep unfolding networks (DUNs), combining conventional iterative optimization algorithms and deep neural networks into a multi-stage framework, have achieved remarkable accomplishments in Image Restoration (IR), such as spectral imaging reconstruction, compressive sensing and super-resolution.It unfolds the iterative optimization steps into a stack of sequentially linked blocks.Each block consists of a Gradient Descent Module (GDM) and a Proximal Mapping Module (PMM) which is equivalent to a denoiser from a Bayesian perspective, operating on Gaussian noise with a known level.However, existing DUNs suffer from two critical limitations: (i) their PMMs share identical architectures and denoising objectives across stages, ignoring the need for stage-specific adaptation to varying noise levels; and (ii) their chain of structurally repetitive blocks results in severe parameter redundancy and high memory consumption, hindering deployment in large-scale or resource-constrained scenarios.To address these challenges, we introduce generalized Deep Low-rank Adaptation (LoRA) Unfolding Networks for image restoration, named LoRun, harmonizing denoising objectives and adapting different denoising levels between stages with compressed memory usage for more efficient DUN.LoRun introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding step.This design decouples the core restoration capability from task-specific adaptation, enabling precise control over denoising intensity without duplicating full network parameters and achieving up to $N$ times parameter reduction for an $N$-stage DUN with on-par or better performance.Extensive experiments conducted on three IR tasks validate the efficiency of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。