将自回归图像模型一步转换为真实图像超分工具,提速10倍且画质领先
VARestorer: One-Step VAR Distillation for Real-World Image Super-Resolution

- 用分布匹配实现单步生成,避免迭代误差累积
- 在DIV2K上达72.32 MUSIQ和0.7669 CLIPIQA,推理快10倍
- 仅微调1.2%参数,适合资源有限的实时超分场景
视觉自回归模型(VAR)在图像生成中表现优异,展现出解决真实世界图像超分辨率(Real-ISR)的潜力。然而,其因果注意力机制限制了对低质量(LQ)输入全局上下文的利用,导致高保真(HQ)输出模糊且不一致;同时,迭代预测中的误差积累严重损害了结果连贯性。为此,我们提出VARestorer,一种简单高效的蒸馏框架,将预训练文本到图像的VAR模型转化为单步超分模型。通过分布匹配,该方法消除迭代优化需求,显著减少误差传播并提升推理速度。此外,引入金字塔图像条件与跨尺度注意力,实现双向尺度交互,充分挖掘输入信息,避免后续低质量标记被忽略。仅通过参数高效适配器微调1.2%的模型参数,即可保持原模型表达能力,大幅提升效率。大量实验表明,VARestorer在DIV2K数据集上达到72.32 MUSIQ和0.7669 CLIPIQA的性能,推理速度相较传统VAR提升10倍。
原文摘要 · Abstract (English)
Recent advancements in visual autoregressive models (VAR) have demonstrated their effectiveness in image generation, highlighting their potential for real-world image super-resolution (Real-ISR). However, adapting VAR for ISR presents critical challenges. The next-scale prediction mechanism, constrained by causal attention, fails to fully exploit global low-quality (LQ) context, resulting in blurry and inconsistent high-quality (HQ) outputs. Additionally, error accumulation in the iterative prediction severely degrades coherence in ISR task. To address these issues, we propose VARestorer, a simple yet effective distillation framework that transforms a pre-trained text-to-image VAR model into a one-step ISR model. By leveraging distribution matching, our method eliminates the need for iterative refinement, significantly reducing error propagation and inference time. Furthermore, we introduce pyramid image conditioning with cross-scale attention, which enables bidirectional scale-wise interactions and fully utilizes the input image information while adapting to the autoregressive mechanism. This prevents later LQ tokens from being overlooked in the transformer. By fine-tuning only 1.2\% of the model parameters through parameter-efficient adapters, our method maintains the expressive power of the original VAR model while significantly enhancing efficiency. Extensive experiments show that VARestorer achieves state-of-the-art performance with 72.32 MUSIQ and 0.7669 CLIPIQA on DIV2K dataset, while accelerating inference by 10 times compared to conventional VAR inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。