提出轻量级网络,高效聚合长距离相似图像特征以提升超分辨率速度与质量。
CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution
- 通过内容感知的令牌聚合机制,跨区域整合相似特征。
- 相比SOTA方法,峰值信噪比最高提升0.33dB,推理速度接近翻倍。
- 适合对实时性与高画质有要求的移动端图像增强场景。
基于Transformer的方法在图像超分辨率(SR)等低层视觉任务中表现优异,但其计算复杂度随空间分辨率呈二次增长。现有方法通过将低分辨率图像划分为局部窗口、轴向条带或膨胀窗口来缓解此问题。然而,这些方法将注意力限制在内容无关的局部区域,难以捕捉长程依赖关系。为此,本文提出轻量级内容感知令牌聚合网络(CATANet)。具体地,设计了一种高效的內容感知令牌聚合模块,通过共享所有图像令牌的中心点并仅在训练阶段更新,实现长程内容相似令牌的聚合,并利用组内自注意力促进长程信息交互,同时引入组间交叉注意力进一步增强全局信息交互。实验表明,相较于最先进的聚类方法SPIN,本方法在性能上更优,最大PSNR提升0.33dB,且推理速度接近翻倍。
原文摘要 · Abstract (English)
Transformer-based methods have demonstrated impressive performance in low-level visual tasks such as Image Super-Resolution (SR). However, its computational complexity grows quadratically with the spatial resolution. A series of works attempt to alleviate this problem by dividing Low-Resolution images into local windows, axial stripes, or dilated windows. SR typically leverages the redundancy of images for reconstruction, and this redundancy appears not only in local regions but also in long-range regions. However, these methods limit attention computation to content-agnostic local regions, limiting directly the ability of attention to capture long-range dependency. To address these issues, we propose a lightweight Content-Aware Token Aggregation Network (CATANet). Specifically, we propose an efficient Content-Aware Token Aggregation module for aggregating long-range content-similar tokens, which shares token centers across all image tokens and updates them only during the training phase. Then we utilize intra-group self-attention to enable long-range information interaction. Moreover, we design an inter-group cross-attention to further enhance global information interaction. The experimental results show that, compared with the state-of-the-art cluster-based method SPIN, our method achieves superior performance, with a maximum PSNR improvement of 0.33dB and nearly double the inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。