提出C2D-ISR框架,让图像超分辨率模型更高效地跨尺度融合特征。
C2D-ISR: Optimizing Attention-based Image Super-resolution from Continuous to Discrete Scales
- 采用连续尺度训练+分层编码,提升多尺度特征学习能力。
- 在SwinIR-L等模型上实现最高0.2dB性能提升和11%计算量降低。
- 适合追求高效率与高精度的图像超分辨率研究者使用。
近年来,注意力机制被用于单图超分辨率(SISR),取得了显著重建效果。然而,现有方法仍受限于简单的训练策略和针对离散放大倍数设计的网络结构,难以有效捕捉多尺度信息。为此,本文提出一种新框架C2D-ISR,从性能与复杂度双重角度优化基于注意力的图像超分辨率模型。该框架采用两阶段训练策略与分层编码机制:连续尺度训练使模型学习尺度间关联与多尺度特征表示;通用化的分层编码可增强空间特征融合与跨尺度信息聚合,显著加快推理速度。我们在三种高效注意力骨干网络(SwinIR-L、SRFormer-L、MambaIRv2-L)上评估该框架,结果表明其相比现有优化框架HiT,在超分辨率性能上最高提升0.2dB,计算复杂度降低最多11%。代码将公开于www.github.com。
原文摘要 · Abstract (English)
In recent years, attention mechanisms have been exploited in single image super-resolution (SISR), achieving impressive reconstruction results. However, these advancements are still limited by the reliance on simple training strategies and network architectures designed for discrete up-sampling scales, which hinder the model's ability to effectively capture information across multiple scales. To address these limitations, we propose a novel framework, \textbf{C2D-ISR}, for optimizing attention-based image super-resolution models from both performance and complexity perspectives. Our approach is based on a two-stage training methodology and a hierarchical encoding mechanism. The new training methodology involves continuous-scale training for discrete scale models, enabling the learning of inter-scale correlations and multi-scale feature representation. In addition, we generalize the hierarchical encoding mechanism with existing attention-based network structures, which can achieve improved spatial feature fusion, cross-scale information aggregation, and more importantly, much faster inference. We have evaluated the C2D-ISR framework based on three efficient attention-based backbones, SwinIR-L, SRFormer-L and MambaIRv2-L, and demonstrated significant improvements over the other existing optimization framework, HiT, in terms of super-resolution performance (up to 0.2dB) and computational complexity reduction (up to 11%). The source code will be made publicly available at www.github.com.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。