一次采样完成多种图像修复，效率与质量双提升
Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration
- 用多退化特征调制+轻量微调，实现单步采样修复
- 在多个数据集上超越现有方法，推理速度更快
- 适合需要快速适配多种退化类型的实用场景
扩散模型在全功能图像修复（AiOIR）中展现出强大潜力，能生成丰富纹理细节。现有方法要么重训练扩散模型，要么通过额外条件引导微调预训练模型，但普遍存在推理成本高、对多种退化类型适应性差的问题。本文提出高效AiOIR方法Diffusion Once and Done（DOD），仅需一次采样即可实现稳定扩散（SD）模型的优越修复效果。首先引入多退化特征调制，利用预训练扩散模型捕捉不同退化提示；随后采用参数高效的低秩适应（LoRA）将提示融合，实现对不同退化类型的快速适配；此外，在SD解码器中集成高保真细节增强模块，进一步提升结构与纹理质量。实验表明，该方法在视觉质量和推理效率方面均优于现有基于扩散模型的修复方法。
原文摘要 · Abstract (English)
Diffusion models have revealed powerful potential in all-in-one image restoration (AiOIR), which is talented in generating abundant texture details. The existing AiOIR methods either retrain a diffusion model or fine-tune the pretrained diffusion model with extra conditional guidance. However, they often suffer from high inference costs and limited adaptability to diverse degradation types. In this paper, we propose an efficient AiOIR method, Diffusion Once and Done (DOD), which aims to achieve superior restoration performance with only one-step sampling of Stable Diffusion (SD) models. Specifically, multi-degradation feature modulation is first introduced to capture different degradation prompts with a pretrained diffusion model. Then, parameter-efficient conditional low-rank adaptation integrates the prompts to enable the fine-tuning of the SD model for adapting to different degradation types. Besides, a high-fidelity detail enhancement module is integrated into the decoder of SD to improve structural and textural details. Experiments demonstrate that our method outperforms existing diffusion-based restoration approaches in both visual quality and inference efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。