arXiv:2604.25457cs.CV2026-04中稿 · the 28th Internati…

用输入图像的视觉特征替代文本描述,提升扩散模型超分辨率精度。

GramSR: Visual Feature Conditioning for Diffusion-Based Super-Resolution

论文配图:GramSR: Visual Feature Conditioning for Diffusion-Based Super-Resolution
图 1 · 摘自论文原文
  • 用DINOv3提取低分辨率图密集视觉特征,替代传统文本条件。
  • 三阶段LoRA依次优化去噪、语义细节和纹理一致性,性能超越现有方法。
  • 支持灵活调节去噪、增强与纹理保留,适合复杂退化场景应用。

尽管近期取得进展,单图超分辨率(SR)在真实世界复杂退化场景中仍具挑战性。基于扩散模型的SR方法,特别是基于Stable Diffusion的方法,虽利用强大生成先验,但通常依赖语义描述的文本条件。这类描述仅提供高层语义,缺乏空间对齐的视觉信息,导致抽象语义与空间细节之间存在表征鸿沟。为此,我们提出GramSR,一种一步式扩散超分辨率框架,以预训练DINOv3编码器从低分辨率输入中提取的密集视觉特征替代文本条件。GramSR采用三阶段LoRA架构,像素级、语义级和纹理级模块依次训练:像素级模块使用ℓ₂损失聚焦去噪,语义级模块通过LPIPS和CSD损失增强感知细节,纹理级模块则通过计算DINOv3特征的格拉姆矩阵损失维持特征相关性一致。推理时,独立引导尺度可灵活控制去噪、语义增强与纹理保留。在标准SR基准上的大量实验表明,GramSR持续优于现有一步式扩散方法,在结构保真度和纹理真实性方面表现更优。代码已公开于:https://github.com/aimagelab/GramSR。

原文摘要 · Abstract (English)

Despite recent advances, single-image super-resolution (SR) remains challenging, especially in real-world scenarios with complex degradations. Diffusion-based SR methods, particularly those built on Stable Diffusion, leverage strong generative priors but commonly rely on text conditioning derived from semantic captioning. Such textual descriptions provide only high-level semantics and lack the spatially aligned visual information required for faithful restoration, leading to a representation gap between abstract semantics and spatially aligned visual details. To address this limitation, we propose GramSR, a one-step diffusion-based SR framework that replaces text conditioning with dense visual features extracted from the low-resolution input using a pre-trained DINOv3 encoder. GramSR adopts a three-stage LoRA architecture, where pixel-level, semantic-level, and texture-level LoRA modules are trained sequentially. The pixel-level module focuses on degradation removal using $\ell_2$ loss, the semantic-level module enhances perceptual details via LPIPS and CSD losses, and the texture-level module enforces feature correlation consistency through a Gram matrix loss computed from DINOv3 features. At inference, independent guidance scales enable flexible control over degradation removal, semantic enhancement, and texture preservation. Extensive experiments on standard SR benchmarks demonstrate that GramSR consistently outperforms existing one-step diffusion-based methods, achieving superior structural fidelity and texture realism. The code for this work is available at: https://github.com/aimagelab/GramSR.

超分辨率扩散模型视觉特征LoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。