用卷积增强注意力机制，让小图像数据更准
CoSwin: Convolution Enhanced Hierarchical Shifted Window Attention For Small-Scale Vision
- 在Transformer中引入可学习局部特征模块，融合局部与全局信息
- 在CIFAR-100和Tiny ImageNet上分别提升4.92%和4.47%
- 适合小规模图像分类任务，尤其数据量少时表现突出
视觉变换器（ViTs）通过自注意力建模长程依赖，在计算机视觉中取得显著成果。然而，其对全局上下文的强调常导致小数据集上局部特征提取能力不足，这主要源于缺乏局部性与平移等变性等关键归纳偏置。为此，我们提出CoSwin，一种新型特征融合架构，将局部卷积特征学习融入分层移位窗口注意力机制。具体而言，CoSwin在每个注意力模块中集成可学习的局部特征增强模块，使模型能同时捕捉精细空间细节与全局语义结构。我们在CIFAR-10、CIFAR-100、MNIST、SVHN和Tiny ImageNet等多个图像分类基准上评估了CoSwin。实验结果表明，其性能持续优于现有卷积与Transformer模型。值得注意的是，CoSwin在CIFAR-10、CIFAR-100、MNIST、SVHN和Tiny ImageNet上分别比基线Swin Transformer提升2.17%、4.92%、0.10%、0.26%和4.47%。这些改进凸显了局部-全局特征融合在提升变压器在小规模视觉任务中泛化性和鲁棒性方面的有效性。代码与预训练权重见https://github.com/puskal-khadka/coswin。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have achieved impressive results in computer vision by leveraging self-attention to model long-range dependencies. However, their emphasis on global context often comes at the expense of local feature extraction in small datasets, particularly due to the lack of key inductive biases such as locality and translation equivariance. To mitigate this, we propose CoSwin, a novel feature-fusion architecture that augments the hierarchical shifted window attention with localized convolutional feature learning. Specifically, CoSwin integrates a learnable local feature enhancement module into each attention block, enabling the model to simultaneously capture fine-grained spatial details and global semantic structure. We evaluate CoSwin on multiple image classification benchmarks including CIFAR-10, CIFAR-100, MNIST, SVHN, and Tiny ImageNet. Our experimental results show consistent performance gains over state-of-the-art convolutional and transformer-based models. Notably, CoSwin achieves improvements of 2.17% on CIFAR-10, 4.92% on CIFAR-100, 0.10% on MNIST, 0.26% on SVHN, and 4.47% on Tiny ImageNet over the baseline Swin Transformer. These improvements underscore the effectiveness of local-global feature fusion in enhancing the generalization and robustness of transformers for small-scale vision. Code and pretrained weights available at https://github.com/puskal-khadka/coswin
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。