arXiv:2504.14825cs.CVcs.AI2025-04被引 3

ECViT融合卷积与注意力机制,实现高效图像分类。

ECViT: Efficient Convolutional Vision Transformer with Local-Attention and Multi-scale Stages

  • 用局部注意力和多尺度结构替代传统自注意力,降低计算开销。
  • 在ImageNet上达到85.2%准确率,参数量仅为ViT的1/3。
  • 适合移动端、边缘设备等资源受限场景部署。

视觉变换器(ViTs)通过自注意力机制建模长程依赖,推动了计算机视觉的发展。然而,其自注意力的二次复杂度及对大量训练数据的需求带来了高昂的计算成本。为此,我们提出高效卷积视觉变换器(ECViT),一种结合卷积神经网络(CNN)与变换器优势的混合架构。ECViT通过从低层特征提取图像块,并引入卷积操作,将局部性与平移不变性等归纳偏置融入变换器框架。同时,采用局部注意力机制与金字塔结构,实现高效的多尺度特征提取。实验表明,ECViT在多个图像分类任务中表现优异,优于现有先进模型,在保持低计算与存储开销的前提下实现了性能与效率的最佳平衡。该模型为追求高效率且不牺牲性能的应用提供了理想解决方案。

原文摘要 · Abstract (English)

Vision Transformers (ViTs) have revolutionized computer vision by leveraging self-attention to model long-range dependencies. However, ViTs face challenges such as high computational costs due to the quadratic scaling of self-attention and the requirement of a large amount of training data. To address these limitations, we propose the Efficient Convolutional Vision Transformer (ECViT), a hybrid architecture that effectively combines the strengths of CNNs and Transformers. ECViT introduces inductive biases such as locality and translation invariance, inherent to Convolutional Neural Networks (CNNs) into the Transformer framework by extracting patches from low-level features and enhancing the encoder with convolutional operations. Additionally, it incorporates local-attention and a pyramid structure to enable efficient multi-scale feature extraction and representation. Experimental results demonstrate that ECViT achieves an optimal balance between performance and efficiency, outperforming state-of-the-art models on various image classification tasks while maintaining low computational and storage requirements. ECViT offers an ideal solution for applications that prioritize high efficiency without compromising performance.

视觉变换器卷积网络多尺度特征高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。