用卷积实现高效图像超分,速度提升3倍且精度更高
Unifying Dimensions: A Linear Adaptive Approach to Lightweight Image Super-Resolution
- 设计线性聚焦可分离注意力,替代自注意力以降低计算开销
- 引入双分支结构与轻量信息交换模块,增强特征融合能力
- 适合追求高速高质图像重建的工业应用或移动端部署
窗口化Transformer在超分辨率任务中表现优异,得益于局部自注意力(SA)带来的自适应建模能力。然而,其计算复杂度和推理延迟高于卷积神经网络。本文指出,Transformer的适应性源于自适应空间聚合与先进结构设计,而高延迟则来自局部自注意力的计算成本及内存布局转换。为此,提出一种基于卷积的线性聚焦可分离注意力(FSA),实现长程动态建模且保持线性复杂度。同时引入双分支结构与超轻量信息交换模块(IEM),增强Token Mixer的信息聚合能力。此外,通过自门控机制改进原有基于空间门控的前馈网络,以保留高维通道信息,支持更复杂的关联建模。基于上述改进,构建了名为线性自适应混合器网络(LAMNet)的卷积型Transformer框架。大量实验表明,LAMNet在性能上优于现有基于SA的Transformer方法,同时保持了卷积网络的计算效率,推理速度提升3倍。代码将公开于:https://github.com/zononhzy/LAMNet。
原文摘要 · Abstract (English)
Window-based transformers have demonstrated outstanding performance in super-resolution tasks due to their adaptive modeling capabilities through local self-attention (SA). However, they exhibit higher computational complexity and inference latency than convolutional neural networks. In this paper, we first identify that the adaptability of the Transformers is derived from their adaptive spatial aggregation and advanced structural design, while their high latency results from the computational costs and memory layout transformations associated with the local SA. To simulate this aggregation approach, we propose an effective convolution-based linear focal separable attention (FSA), allowing for long-range dynamic modeling with linear complexity. Additionally, we introduce an effective dual-branch structure combined with an ultra-lightweight information exchange module (IEM) to enhance the aggregation of information by the Token Mixer. Finally, with respect to the structure, we modify the existing spatial-gate-based feedforward neural networks by incorporating a self-gate mechanism to preserve high-dimensional channel information, enabling the modeling of more complex relationships. With these advancements, we construct a convolution-based Transformer framework named the linear adaptive mixer network (LAMNet). Extensive experiments demonstrate that LAMNet achieves better performance than existing SA-based Transformer methods while maintaining the computational efficiency of convolutional neural networks, which can achieve a \(3\times\) speedup of inference time. The code will be publicly available at: https://github.com/zononhzy/LAMNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。