用两次上采样提升图像超分辨率,生成2048×2048高清图
TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution

- 分两阶段训练:先在低分辨率训练,再用分块循环策略提升至高分辨率
- 在1024²和2048²分辨率下均超越现有方法,生成质量显著提升
- 适合作为轻量级设备上的高分辨率图像生成方案
基于扩散的生成模型在真实世界图像超分辨率(SR)中取得显著进展。借助分块扩散技术,这些模型可生成超出其原生支持分辨率的高分辨率图像。然而,当输出分辨率高达2048²时,图像质量仍极差,主要原因在于:上采样倍率(如×8)超过模型原生支持的上采样倍率(如×4),且受限于模型原生支持的分辨率。实际中,训练原生高分辨率模型需更大架构,带来巨大计算开销和显存成本,难以在资源有限设备上运行。为此,我们提出TUDSR——一种两次上采样扩散框架,用于更高阶超分辨率。TUDSR包含两个阶段:第一阶段在R分辨率下训练;第二阶段引入循环分块训练策略,在NR分辨率下优化。每阶段采用单步GAN结构,含生成器与判别器。基于SD2.1-base,我们构建TUDSR-S,在多个基准上达到当前最佳性能。大量实验表明,TUDSR-S可在1024²乃至2048²分辨率下生成高质量图像,显著优于现有方法。代码已公开于https://github.com/wuer5/TUDSR。
原文摘要 · Abstract (English)
Diffusion-based generative models have achieved remarkable success in real-world image super-resolution (SR). With tiled diffusion techniques, these models can produce high-resolution images that exceed their native-supported resolution. However, the quality of such high-resolution (e.g $2048^2$) outputs often remains extremely poor, primarily due to two factors we consider: the image upsampling ratio (e.g $\times8$) exceeding the model's native-supported upsampling ratio (e.g $\times4$), and the model's native-supported resolution. In practice, training a native high-resolution model requires larger architectures, which incur significant computational overhead and GPU memory costs, making it hard on limited-resource equipment. Thus, we present TUDSR, a Twice Upsampling-Diffusion framework for higher SR. The TUDSR framework mainly consists of two stages: the first involves training at $R$-resolution, and the second introduces a looped chunk-based training strategy at $NR$-resolution. Each stage adapts a one-step GAN architecture comprising a generator and a discriminator. Based on SD2.1-base, we develop TUDSR-S, which achieves state-of-the-art performance across multiple benchmarks. Extensive experiments further demonstrate that TUDSR-S generates high-quality images at the resolutions of $1024^2$ and even $2048^2$, significantly outperforming existing approaches. Code is available at https://github.com/wuer5/TUDSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。