融合CNN与Transformer，提升小数据集上的特征提取效率。
CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction
- 设计轻量级模块，协同捕捉局部与全局特征。
- 在少于10万样本数据集上达86.76%准确率，参数仅2032万。
- 适合资源受限场景下的高效视觉任务部署。
卷积神经网络（CNN）和视觉变换器（ViTs）在计算机视觉中分别擅长局部与全局特征提取。然而，现有方法在融合两者时常导致效率低下。为此，本文提出CNN-Transformer聚合网络（CTA-Net），结合变压器捕捉长程依赖、CNN提取局部特征，实现对细节与上下文信息的高效处理。CTA-Net引入轻量级多尺度特征融合多头自注意力（LMF-MHSA）模块，有效整合多尺度特征且减少参数量；同时通过反向重建CNN变体（RRCV）模块增强CNN嵌入变换器架构的能力。在样本数少于10万的小规模数据集上，实验表明，CTA-Net达到TOP-1准确率86.76%，参数量仅20.32M，计算量为2.83B FLOPs，展现出卓越的性能与高效性，是小规模视觉任务的轻量化高效解决方案。
原文摘要 · Abstract (English)
Convolutional neural networks (CNNs) and vision transformers (ViTs) have become essential in computer vision for local and global feature extraction. However, aggregating these architectures in existing methods often results in inefficiencies. To address this, the CNN-Transformer Aggregation Network (CTA-Net) was developed. CTA-Net combines CNNs and ViTs, with transformers capturing long-range dependencies and CNNs extracting localized features. This integration enables efficient processing of detailed local and broader contextual information. CTA-Net introduces the Light Weight Multi-Scale Feature Fusion Multi-Head Self-Attention (LMF-MHSA) module for effective multi-scale feature integration with reduced parameters. Additionally, the Reverse Reconstruction CNN-Variants (RRCV) module enhances the embedding of CNNs within the transformer architecture. Extensive experiments on small-scale datasets with fewer than 100,000 samples show that CTA-Net achieves superior performance (TOP-1 Acc 86.76\%), fewer parameters (20.32M), and greater efficiency (FLOPs 2.83B), making it a highly efficient and lightweight solution for visual tasks on small-scale datasets (fewer than 100,000).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。