提出首个模仿人眼先看整体再聚焦细节的纯卷积主干网络
OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels
- 设计分枝结构：先生成全局概览，再聚焦细节感知
- 引入动态上下文混合卷积，有效建模长程依赖且保持局部特性
- 在精度和效率上全面超越同类模型，适合视觉任务新架构研究
人类视觉系统中，大脑会先快速获取场景的整体概览以发现显著线索（即‘先看整体’），随后进行更精细的局部观察（即‘再仔细看’）。然而，现代卷积神经网络仍采用逐层下采样的金字塔结构来扩大感受野，忽略了这一关键的生物启发原则。本文提出 OverLoCK，首个明确融合自上而下注意力机制的纯卷积主干网络。其采用三子网协同的分支结构：1）基础网络编码低/中层特征；2）轻量级概览网络通过粗粒度全局上下文建模生成动态自上而下注意力（即‘先看整体’）；3）稳健聚焦网络在注意力引导下执行更精细的感知（即‘再仔细看’）。为充分发挥注意力潜力，我们提出一种新颖的上下文混合动态卷积（ContMix），在输入分辨率提升时仍能有效建模长程依赖并保留固有局部归纳偏置。OverLoCK 在多项任务中表现卓越：OverLoCK-T 的 Top-1 准确率达 84.2%，显著优于 ConvNeXt-B，且仅需其约三分之一的计算量与参数。在目标检测中，OverLoCK-S 比 MogaNet-B 高出 1% 的 AP^b。在语义分割中，OverLoCK-T 比 UniRepLKNet-T 提升 1.7% 的 mIoU。代码已开源。
原文摘要 · Abstract (English)
Top-down attention plays a crucial role in the human vision system, wherein the brain initially obtains a rough overview of a scene to discover salient cues (i.e., overview first), followed by a more careful finer-grained examination (i.e., look closely next). However, modern ConvNets remain confined to a pyramid structure that successively downsamples the feature map for receptive field expansion, neglecting this crucial biomimetic principle. We present OverLoCK, the first pure ConvNet backbone architecture that explicitly incorporates a top-down attention mechanism. Unlike pyramid backbone networks, our design features a branched architecture with three synergistic sub-networks: 1) a Base-Net that encodes low/mid-level features; 2) a lightweight Overview-Net that generates dynamic top-down attention through coarse global context modeling (i.e., overview first); and 3) a robust Focus-Net that performs finer-grained perception guided by top-down attention (i.e., look closely next). To fully unleash the power of top-down attention, we further propose a novel context-mixing dynamic convolution (ContMix) that effectively models long-range dependencies while preserving inherent local inductive biases even when the input resolution increases, addressing critical limitations in existing convolutions. Our OverLoCK exhibits a notable performance improvement over existing methods. For instance, OverLoCK-T achieves a Top-1 accuracy of 84.2%, significantly surpassing ConvNeXt-B while using only around one-third of the FLOPs/parameters. On object detection, our OverLoCK-S clearly surpasses MogaNet-B by 1% in AP^b. On semantic segmentation, our OverLoCK-T remarkably improves UniRepLKNet-T by 1.7% in mIoU. Code is publicly available at https://github.com/LMMMEng/OverLoCK.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。