arXiv:2602.05262cs.CV2026-02

轻量级模型ReGLA高效建模大视野,兼顾精度与速度。

ReGLA: Efficient Receptive-Field Modeling with Gated Linear Attention Network

  • 用门控线性注意力+高效卷积实现全局建模
  • 512px输入下仅4.98毫秒延迟,准确率达80.85%
  • 适合高分辨率图像任务,如检测与分割

在高分辨率图像上平衡精度与延迟是轻量级模型的关键挑战,尤其对基于Transformer的架构而言,其通常存在延迟过高的问题。为此,我们提出一系列轻量级混合网络ReGLA,将高效的局部特征提取卷积与基于ReLU的门控线性注意力相结合,用于全局建模。设计包含三项创新:用于提升卷积效率并保持大感受野的高效大感受野(ELRF)模块;在保持线性复杂度的同时增强局部特征表示的ReLU门控调制注意力(RGMA)模块;以及多教师蒸馏策略,以提升下游任务性能。大量实验验证了ReGLA的优势:其中ReGLA-M在ImageNet-1K上224px输入下达到80.85%的Top-1准确率,512px输入下延迟仅为4.98毫秒。此外,其在下游任务中表现更优,于COCO目标检测上取得3.1%的AP提升,于ADE20K语义分割上取得3.6%的mIoU提升,成为高分辨率视觉应用的前沿解决方案。

原文摘要 · Abstract (English)

Balancing accuracy and latency on high-resolution images is a critical challenge for lightweight models, particularly for Transformer-based architectures that often suffer from excessive latency. To address this issue, we introduce \textbf{ReGLA}, a series of lightweight hybrid networks, which integrates efficient convolutions for local feature extraction with ReLU-based gated linear attention for global modeling. The design incorporates three key innovations: the Efficient Large Receptive Field (ELRF) module for enhancing convolutional efficiency while preserving a large receptive field; the ReLU Gated Modulated Attention (RGMA) module for maintaining linear complexity while enhancing local feature representation; and a multi-teacher distillation strategy to boost performance on downstream tasks. Extensive experiments validate the superiority of ReGLA; particularly the ReGLA-M achieves \textbf{80.85\%} Top-1 accuracy on ImageNet-1K at $224px$, with only \textbf{4.98 ms} latency at $512px$. Furthermore, ReGLA outperforms similarly scaled iFormer models in downstream tasks, achieving gains of \textbf{3.1\%} AP on COCO object detection and \textbf{3.6\%} mIoU on ADE20K semantic segmentation, establishing it as a state-of-the-art solution for high-resolution visual applications.

轻量模型注意力机制图像分类高分辨率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。