用自回归模型实现快速高保真图像超分辨率，抗退化干扰更强。
NSARM: Next-Scale Autoregressive Modeling for Robust Real-World Image Super-Resolution
- 分两阶段训练：先映射低质图到初步尺度，再端到端微调增强鲁棒性
- 在多个真实退化数据集上超越现有方法，速度比扩散模型快10倍以上
- 适合对输入质量不稳定的图像超分任务，尤其适合实时应用
当前真实世界图像超分辨率（Real-ISR）方法多依赖预训练文本到图像扩散模型，要么从随机噪声生成（结果逼真但慢），要么直接从低质图生成（快但质量差）。这类方法通常仅微调ControlNet或LoRA，固定主模型，易产生过增强伪影和幻觉，对不同退化程度输入泛化能力弱。近期视觉自回归模型如Infinity可实现强生成能力并提升效率，采用位级下一尺度预测策略。本文提出新型框架NSARM，基于下一尺度预测思想，通过两阶段训练：先训练变换网络将输入低质图映射至初步尺度，再进行端到端全模型微调。该设计在保持生成能力的同时显著提升对多种退化输入的鲁棒性。大量定量与定性评估表明，作为纯自回归模型，NSARM在多个真实退化数据集（包括RealSR、REDS4、Clipper）上优于现有方法，视觉效果更优且推理速度快于扩散模型10倍以上，展现出更强的泛化能力。项目主页：https://github.com/Xiangtaokong/NSARM
原文摘要 · Abstract (English)
Most recent real-world image super-resolution (Real-ISR) methods employ pre-trained text-to-image (T2I) diffusion models to synthesize the high-quality image either from random Gaussian noise, which yields realistic results but is slow due to iterative denoising, or directly from the input low-quality image, which is efficient but at the price of lower output quality. These approaches train ControlNet or LoRA modules while keeping the pre-trained model fixed, which often introduces over-enhanced artifacts and hallucinations, suffering from the robustness to inputs of varying degradations. Recent visual autoregressive (AR) models, such as pre-trained Infinity, can provide strong T2I generation capabilities while offering superior efficiency by using the bitwise next-scale prediction strategy. Building upon next-scale prediction, we introduce a robust Real-ISR framework, namely Next-Scale Autoregressive Modeling (NSARM). Specifically, we train NSARM in two stages: a transformation network is first trained to map the input low-quality image to preliminary scales, followed by an end-to-end full-model fine-tuning. Such a comprehensive fine-tuning enhances the robustness of NSARM in Real-ISR tasks without compromising its generative capability. Extensive quantitative and qualitative evaluations demonstrate that as a pure AR model, NSARM achieves superior visual results over existing Real-ISR methods while maintaining a fast inference speed. Most importantly, it demonstrates much higher robustness to the quality of input images, showing stronger generalization performance. Project page: https://github.com/Xiangtaokong/NSARM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。