用分层提示增强轻量图像超分辨率,突破感受野限制
PromptSR: Cascade Prompting for Lightweight Image Super-Resolution
- 设计级联提示模块,分三步提升全局与局部特征融合
- 在保持低计算量前提下,实现比现有方法更高的图像重建质量
- 适合移动端、边缘设备部署,尤其关注高效图像增强的开发者
尽管轻量级视觉变换器显著推进了图像超分辨率(SR),但其受限于窗口化自注意力建模带来的有限感受野。窗口大小相关的二次计算复杂度限制了大窗口使用,难以在低计算成本下扩展感受野。为此,本文提出PromptSR,一种新型轻量级图像超分辨率方法。核心为级联提示块(CPB),包含三个级联提示层:全局锚点提示层(GAPL)和两个局部提示层(LPL)。GAPL利用下采样特征作为锚点,通过跨尺度注意力构建低维锚点提示(APs),大幅降低计算开销;这些具备增强全局感知能力的APs作为全局提示,高效促进长距离标记连接。随后两个LPL结合类别自注意力与窗口自注意力,实现由粗到精的表示细化。它们以GAPL生成的注意力图为额外全局提示,使模型在不同粒度下感知全局特征,实现自适应局部优化。该设计有效融合全局先验与局部细节,在保持低计算量的同时显著扩大感受野。实验表明,该方法在定量、定性和复杂度评估中均优于现有先进轻量级SR方法。代码将公开于https://github.com/wenyang001/PromptSR。
原文摘要 · Abstract (English)
Although the lightweight Vision Transformer has significantly advanced image super-resolution (SR), it faces the inherent challenge of a limited receptive field due to the window-based self-attention modeling. The quadratic computational complexity relative to window size restricts its ability to use a large window size for expanding the receptive field while maintaining low computational costs. To address this challenge, we propose PromptSR, a novel prompt-empowered lightweight image SR method. The core component is the proposed cascade prompting block (CPB), which enhances global information access and local refinement via three cascaded prompting layers: a global anchor prompting layer (GAPL) and two local prompting layers (LPLs). The GAPL leverages downscaled features as anchors to construct low-dimensional anchor prompts (APs) through cross-scale attention, significantly reducing computational costs. These APs, with enhanced global perception, are then used to provide global prompts, efficiently facilitating long-range token connections. The two LPLs subsequently combine category-based self-attention and window-based self-attention to refine the representation in a coarse-to-fine manner. They leverage attention maps from the GAPL as additional global prompts, enabling them to perceive features globally at different granularities for adaptive local refinement. In this way, the proposed CPB effectively combines global priors and local details, significantly enlarging the receptive field while maintaining the low computational costs of our PromptSR. The experimental results demonstrate the superiority of our method, which outperforms state-of-the-art lightweight SR methods in quantitative, qualitative, and complexity evaluations. Our code will be released at https://github.com/wenyang001/PromptSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。