用知识蒸馏让图像超分模型一步完成,效果更好且更省资源。
One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation
- 通过训练学生网络生成数据,使假的残差移位模型与教师模型一致。
- 单步恢复性能超越教师模型,在感知质量上显著提升(LPIPS降低0.12)。
- 适合追求高效高质超分的开发者,尤其适合资源受限场景。
基于扩散模型的超分辨率(SR)虽能生成高质量图像,但计算成本高昂。尽管已有多种加速方法,如SinSR无法还原真实细节,而OSEDiff可能产生不存在的结构。为此,本文提出一种新的知识蒸馏方法RSD,用于残差移位(ResShift)。该方法训练学生网络生成图像,使得在这些图像上训练的伪残差移位模型能与教师模型完全对齐。RSD实现单步恢复,且在多种感知指标(LPIPS、CLIPIQA、MUSIQ)上显著优于教师模型。实验表明,RSD超越同为蒸馏方法的SinSR,达到当前最优扩散超分蒸馏方法的水平,同时保持较低计算开销。相比基于预训练文生图模型的方法,RSD在感知质量上具有竞争力,且参数量、显存占用和训练成本更低。实验覆盖RealSR、RealSet65、DRealSR、ImageNet和DIV2K等真实世界与合成数据集。代码已开源:https://github.com/Daniil-Selikhanovych/RSD。
原文摘要 · Abstract (English)
Diffusion models for super-resolution (SR) produce high-quality visual results but require expensive computational costs. Despite the development of several methods to accelerate diffusion-based SR models, some (e.g., SinSR) fail to produce realistic perceptual details, while others (e.g., OSEDiff) may hallucinate non-existent structures. To overcome these issues, we present RSD, a new distillation method for ResShift. Our method is based on training the student network to produce images such that a new fake ResShift model trained on them will coincide with the teacher model. RSD achieves single-step restoration and outperforms the teacher by a noticeable margin in various perceptual metrics (LPIPS, CLIPIQA, MUSIQ). We show that our distillation method can surpass SinSR, the other distillation-based method for ResShift, making it on par with state-of-the-art diffusion SR distillation methods with limited computational costs in terms of perceptual quality. Compared to SR methods based on pre-trained text-to-image models, RSD produces competitive perceptual quality and requires fewer parameters, GPU memory, and training cost. We provide experimental results on various real-world and synthetic datasets, including RealSR, RealSet65, DRealSR, ImageNet, and DIV2K. We provide the code at https://github.com/Daniil-Selikhanovych/RSD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。