arXiv:2410.08049cs.CVcs.AI2024-10TPAMI被引 28

用大卷积核设计更高效通用的图像模型,性能超越传统CNN和Transformer。

Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations

  • 用少数大卷积核替代多层小卷积核,提升空间信息捕捉能力。
  • ImageNet准确率88.0%,在多个任务上优于前代模型,推理更快。
  • 适合追求高效率与跨模态通用性的视觉研究者使用。

本文提出在现代卷积神经网络(ConvNets)中采用大卷积核的设计范式。研究表明,使用少量大卷积核而非堆叠多个小卷积核,是一种更优的设计策略。我们提出了适用于大卷积核卷积网络的一系列架构设计准则,以优化其效率与性能。为此,我们构建了UniRepLKNet架构,系统性地针对大卷积核的特点进行设计,强调其无需深层堆叠即可捕获广泛空间信息的能力。该模型在ImageNet上达到88.0%的准确率,ADE20K上实现55.6%的mIoU,COCO目标检测框AP达56.4%,且在时间序列预测、音频、点云和视频识别等多模态任务中表现优异,展现出强大的通用建模能力,同时推理速度优于视觉变压器。实验表明,大卷积核网络具有更大的有效感受野和更高的形状偏置,摆脱了小卷积核常见的纹理偏置。所有代码与模型已公开于https://github.com/AILab-CVC/UniRepLKNet,促进社区研究发展。

原文摘要 · Abstract (English)

This paper proposes the paradigm of large convolutional kernels in designing modern Convolutional Neural Networks (ConvNets). We establish that employing a few large kernels, instead of stacking multiple smaller ones, can be a superior design strategy. Our work introduces a set of architecture design guidelines for large-kernel ConvNets that optimize their efficiency and performance. We propose the UniRepLKNet architecture, which offers systematical architecture design principles specifically crafted for large-kernel ConvNets, emphasizing their unique ability to capture extensive spatial information without deep layer stacking. This results in a model that not only surpasses its predecessors with an ImageNet accuracy of 88.0%, an ADE20K mIoU of 55.6%, and a COCO box AP of 56.4% but also demonstrates impressive scalability and performance on various modalities such as time-series forecasting, audio, point cloud, and video recognition. These results indicate the universal modeling abilities of large-kernel ConvNets with faster inference speed compared with vision transformers. Our findings reveal that large-kernel ConvNets possess larger effective receptive fields and a higher shape bias, moving away from the texture bias typical of smaller-kernel CNNs. All codes and models are publicly available at https://github.com/AILab-CVC/UniRepLKNet promoting further research and development in the community.

大卷积核视觉通用性模型效率UniRepLKNet

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。