arXiv:2501.18993cs.CV2025-01ICML被引 37

用自回归框架提升图像超分,兼顾清晰度与真实感。

Visual Autoregressive Modeling for Image Super-Resolution

  • 通过下一尺度预测建模,结合前缀标记保留语义信息
  • 生成图像在保真度和真实感上均优于扩散模型,效率更高
  • 适合追求高质量图像生成的视觉任务研究者

图像超分辨率(ISR)因生成模型的兴起取得显著进展。然而,保真度与真实感之间的权衡、计算复杂度等问题仍限制其应用。受语言领域自回归模型成功的启发,我们提出一种新型视觉自回归框架VARSR,采用下一尺度预测形式。为有效整合并保留低分辨率图像中的语义信息,引入前缀标记作为条件输入;采用尺度对齐的旋转位置编码捕捉空间结构;利用扩散精修器建模量化残差损失以实现像素级保真度;提出基于图像的无分类器引导策略,提升生成图像的真实感。此外,我们构建了大规模数据集并设计训练流程,获得稳健的生成先验。定量与定性结果表明,VARSR在生成高保真、高真实感图像方面表现优异,且比基于扩散的方法更具效率。代码将开源于https://github.com/qyp2000/VARSR。

原文摘要 · Abstract (English)

Image Super-Resolution (ISR) has seen significant progress with the introduction of remarkable generative models. However, challenges such as the trade-off issues between fidelity and realism, as well as computational complexity, have also posed limitations on their application. Building upon the tremendous success of autoregressive models in the language domain, we propose \textbf{VARSR}, a novel visual autoregressive modeling for ISR framework with the form of next-scale prediction. To effectively integrate and preserve semantic information in low-resolution images, we propose using prefix tokens to incorporate the condition. Scale-aligned Rotary Positional Encodings are introduced to capture spatial structures and the diffusion refiner is utilized for modeling quantization residual loss to achieve pixel-level fidelity. Image-based Classifier-free Guidance is proposed to guide the generation of more realistic images. Furthermore, we collect large-scale data and design a training process to obtain robust generative priors. Quantitative and qualitative results show that VARSR is capable of generating high-fidelity and high-realism images with more efficiency than diffusion-based methods. Our codes will be released at https://github.com/qyp2000/VARSR.

图像超分自回归生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。