用对抗压缩提升真实图像超分速度,快9.3倍且质量不降
Adversarial Diffusion Compression for Real-World Image Super-Resolution
- 将单步扩散模型蒸馏为轻量级扩散GAN,减少73%推理时间
- 通过对抗蒸馏和预训练解码器,保留生成能力同时降低78%计算量
- 适合需要快速部署的图像增强场景,如手机端实时修复
真实世界图像超分辨率(Real-ISR)旨在从由复杂未知过程退化的低分辨率输入中重建高分辨率图像。尽管基于Stable Diffusion(SD)的方法取得显著进展,其多步推理速度慢,难以实际部署。近期的一步式网络如OSEDiff和S3Diff虽缓解此问题,但仍依赖大型预训练SD模型,导致计算开销高。本文提出AdcSR方法,在对抗扩散压缩(ADC)框架下,将一步扩散网络OSEDiff蒸馏为轻量级扩散-GAN模型。我们细致分析OSEDiff模块,分为可移除(如VAE编码器、提示提取器、文本编码器等)与可剪枝(去噪UNet和VAE解码器)。为避免性能下降,我们对剪枝后的VAE解码器进行预训练以恢复解码能力,并采用对抗蒸馏补偿损失。该设计使推理时间减少73%,计算量降低78%,参数量减少74%,同时保持生成能力。实验表明,AdcSR在合成与真实数据集上均达到竞争力的恢复质量,相较先前一步扩散方法提速高达9.3倍。代码与模型见https://github.com/Guaishou74851/AdcSR。
原文摘要 · Abstract (English)
Real-world image super-resolution (Real-ISR) aims to reconstruct high-resolution images from low-resolution inputs degraded by complex, unknown processes. While many Stable Diffusion (SD)-based Real-ISR methods have achieved remarkable success, their slow, multi-step inference hinders practical deployment. Recent SD-based one-step networks like OSEDiff and S3Diff alleviate this issue but still incur high computational costs due to their reliance on large pretrained SD models. This paper proposes a novel Real-ISR method, AdcSR, by distilling the one-step diffusion network OSEDiff into a streamlined diffusion-GAN model under our Adversarial Diffusion Compression (ADC) framework. We meticulously examine the modules of OSEDiff, categorizing them into two types: (1) Removable (VAE encoder, prompt extractor, text encoder, etc.) and (2) Prunable (denoising UNet and VAE decoder). Since direct removal and pruning can degrade the model's generation capability, we pretrain our pruned VAE decoder to restore its ability to decode images and employ adversarial distillation to compensate for performance loss. This ADC-based diffusion-GAN hybrid design effectively reduces complexity by 73% in inference time, 78% in computation, and 74% in parameters, while preserving the model's generation capability. Experiments manifest that our proposed AdcSR achieves competitive recovery quality on both synthetic and real-world datasets, offering up to 9.3$\times$ speedup over previous one-step diffusion-based methods. Code and models are available at https://github.com/Guaishou74851/AdcSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。