arXiv:2604.03225cs.CV2026-04中稿 · CVPR被引 2

纯视觉模型实现高质量图像超分辨率,训练成本低于十分之一。

VOSR: A Vision-Only Generative Model for Image Super-Resolution

  • 用视觉编码器提取低分辨率输入的语义特征,指导生成过程。
  • 新设计的恢复导向引导策略减少幻觉,结构更忠实于原图。
  • 无需多模态预训练,适合追求高效与真实性的超分应用。

现有生成式图像超分辨率方法大多依赖在海量图文数据上预训练的文本到图像扩散模型。尽管有效,但其起点是通用图文生成器,而超分辨率本质上是基于低分辨率输入的图像修复任务。本文提出纯视觉生成框架VOSR,首先利用预训练视觉编码器从低分辨率输入中提取语义丰富且空间对齐的特征作为视觉语义引导。针对从零训练的生成模型,重新审视无分类器引导机制,发现标准无条件分支不适用于修复任务,因此引入保留弱低分辨率锚点的修复导向引导策略。基于此设计,我们从零训练多步VOSR模型,并蒸馏为单步模型以实现高效推理。VOSR训练成本不足代表性图文基方法的十分之一,在多步与单步设置下均达到竞争力甚至更优的感知质量与效率,且在合成与真实世界基准上产生更忠实的结构、更少的幻觉。首次证明高质生成式超分辨率可在无多模态预训练条件下实现。代码与模型见https://github.com/cswry/VOSR。

原文摘要 · Abstract (English)

Most of the recent generative image super-resolution (SR) methods rely on adapting large text-to-image (T2I) diffusion models pretrained on web-scale text-image data. While effective, this paradigm starts from a generic T2I generator, despite that SR is fundamentally a low-resolution (LR) input-conditioned image restoration task. In this work, we investigate whether an SR model trained purely on visual data can rival T2I-based ones. To this end, we propose VOSR, a Vision-Only generative framework for SR. We first extract semantically rich and spatially grounded features from the LR input using a pretrained vision encoder as visual semantic guidance. We then revisit classifier-free guidance for training generative models and show that the standard unconditional branch is ill-suited to restoration models trained from scratch. We therefore replace it with a restoration-oriented guidance strategy that preserves weak LR anchors. Built upon these designs, we first train a multi-step VOSR model from scratch and then distill it into a one-step model for efficient inference. VOSR requires less than one-tenth of the training cost of representative T2I-based SR methods, yet in both multi-step and one-step settings, it achieves competitive or even better perceptual quality and efficiency, while producing more faithful structures with fewer hallucinations on both synthetic and real-world benchmarks. Our results, for the first time, show that high-quality generative SR can be achieved without multimodal pretraining. The code and models can be found at https://github.com/cswry/VOSR.

图像超分生成模型纯视觉扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。