CUST通过聚类相似块实现轻量超分辨率,兼顾全局与局部信息。
CUST: Clustered Unit-level Similarity Transformer for Lightweight Image Super-Resolution

- 基于聚类的单元级相似性注意力,扩展上下文感知范围。
- 相比同类模型,内存占用更低、推理速度更快,性能不降。
- 适合资源受限场景下的图像超分应用。
近期基于视觉变压器(ViT)的模型在图像超分辨率任务中表现卓越,但其空间分辨率相关的二次计算复杂度严重制约了效率,导致高延迟和大量内存消耗。为缓解此问题,已有多种基于窗口的注意力机制被提出,但它们本质上削弱了ViT擅长的长距离依赖建模能力。为此,本文提出聚类单元级相似性变压器(CUST),一种高效融合全局与局部信息的新架构。具体而言,CUST使每个图像块能在扩大区域范围内聚合并关注相似块,从而捕获广泛上下文理解;同时,采用重叠注意力窗口捕捉局部依赖,并通过计算原始特征与其下采样-上采样复现版本之间的残差差异,显式提取高频细节。大量实验表明,所提模型在实际约束下实现了计算效率与重建性能的实用平衡,相比近期全局上下文或轻量级模型,具有更小内存开销和更快推理速度。代码已开源。
原文摘要 · Abstract (English)
Recently, Vision Transformer (ViT)-based models have exhibited remarkable performance in image super-resolution. However, the quadratic computational complexity of ViTs with respect to spatial resolution severely constrains their efficiency, leading to high latency and massive memory consumption. To alleviate this, various window-based attention mechanisms have been proposed; yet, they inherently compromise the long-range dependency modeling that is the primary advantage of ViTs. To overcome these limitations, we propose the Clustered Unit-level Similarity Transformer (CUST), a novel architecture that efficiently integrates global and local information. Specifically, CUST enables each patch to aggregate and attend to similar patches within a broadened regional scope outside its local window, thereby capturing extensive contextual understanding. Furthermore, it employs overlapping attention windows to capture local dependencies, while explicitly extracting high-frequency details by computing the residual difference between the original features and their downsampled-upsampled counterparts. Comprehensive experiments demonstrate that our proposed model achieves a practical balance between computational efficiency and restoration performance. It achieves a lower memory footprint and faster inference speed compared to recent global context or lightweight models under realistic constraints. Code is available at [https://github.com/jwgdmkj/CUST].
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。