提出大小卷积协同机制,让轻量模型更高效感知与聚焦。
LSNet: See Large, Focus Small
- 用大核感知+小核聚合的LS卷积,模拟人眼视觉动态能力。
- 在多个任务中超越现有轻量模型,性能效率双提升。
- 适合资源受限场景下的实时视觉应用,如移动端部署。
视觉网络设计(如卷积神经网络和视觉变换器)显著推动了计算机视觉发展,但其复杂计算限制了实际部署,尤其在实时应用中。现有轻量模型多依赖自注意力和卷积进行标记混合,这种依赖在感知与特征聚合过程中存在效率与效果的瓶颈,难以在有限算力下平衡性能与效率。本文受高效人眼视觉系统启发,提出“看大、聚焦小”的轻量级视觉网络设计策略。引入LS(Large-Small)卷积,结合大核感知与小核聚合,能高效捕捉广泛感知信息并实现对动态复杂视觉表征的精确特征聚合,从而实现高效视觉信息处理。基于此,我们提出LSNet这一新型轻量级模型系列。大量实验表明,LSNet在多种视觉任务中均优于现有轻量级网络。代码与模型已开源:https://github.com/jameslahm/lsnet。
原文摘要 · Abstract (English)
Vision network designs, including Convolutional Neural Networks and Vision Transformers, have significantly advanced the field of computer vision. Yet, their complex computations pose challenges for practical deployments, particularly in real-time applications. To tackle this issue, researchers have explored various lightweight and efficient network designs. However, existing lightweight models predominantly leverage self-attention mechanisms and convolutions for token mixing. This dependence brings limitations in effectiveness and efficiency in the perception and aggregation processes of lightweight networks, hindering the balance between performance and efficiency under limited computational budgets. In this paper, we draw inspiration from the dynamic heteroscale vision ability inherent in the efficient human vision system and propose a ``See Large, Focus Small'' strategy for lightweight vision network design. We introduce LS (\textbf{L}arge-\textbf{S}mall) convolution, which combines large-kernel perception and small-kernel aggregation. It can efficiently capture a wide range of perceptual information and achieve precise feature aggregation for dynamic and complex visual representations, thus enabling proficient processing of visual information. Based on LS convolution, we present LSNet, a new family of lightweight models. Extensive experiments demonstrate that LSNet achieves superior performance and efficiency over existing lightweight networks in various vision tasks. Codes and models are available at https://github.com/jameslahm/lsnet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。