对比视觉Transformer与传统CNN,发现前者在全局理解上更优,尤其适合医学图像分析。
Hands-on Evaluation of Visual Transformers for Object Recognition and Detection
- 采用自注意力机制的ViT模型能捕捉图像全局关系,优于依赖局部特征的CNN。
- Swin和CvT等混合/分层结构在准确率与计算成本间取得平衡,医疗数据上提升显著。
- 数据增强可大幅提升医学图像中Swin Transformer性能,适合医疗视觉任务。
卷积神经网络(CNN)在计算机视觉中常受限于局部模式关注,难以把握图像全局语义。相比之下,受语言模型启发的视觉变压器(ViT)利用自注意力机制,能够建模整个图像的长程依赖。本文系统比较了纯ViT、分层及混合型ViT与传统CNN在图像分类、目标检测和医学图像分类等任务上的表现。实验基于ImageNet图像分类和COCO目标检测标准数据集,并在胸部X光数据集ChestX-ray14上测试医学应用。结果表明,混合与分层结构的ViT(如Swin Transformer和CvT)在精度与计算资源之间实现了良好权衡。此外,在医学图像上通过数据增强技术进一步提升了模型性能,尤其是Swin Transformer效果突出。整体来看,视觉变压器在需要全局上下文理解的任务中表现出色,已超越或媲美传统CNN,尤其在医疗影像分析中具有显著优势。
原文摘要 · Abstract (English)
Convolutional Neural Networks (CNNs) for computer vision sometimes struggle with understanding images in a global context, as they mainly focus on local patterns. On the other hand, Vision Transformers (ViTs), inspired by models originally created for language processing, use self-attention mechanisms, which allow them to understand relationships across the entire image. In this paper, we compare different types of ViTs (pure, hierarchical, and hybrid) against traditional CNN models across various tasks, including object recognition, detection, and medical image classification. We conduct thorough tests on standard datasets like ImageNet for image classification and COCO for object detection. Additionally, we apply these models to medical imaging using the ChestX-ray14 dataset. We find that hybrid and hierarchical transformers, especially Swin and CvT, offer a strong balance between accuracy and computational resources. Furthermore, by experimenting with data augmentation techniques on medical images, we discover significant performance improvements, particularly with the Swin Transformer model. Overall, our results indicate that Vision Transformers are competitive and, in many cases, outperform traditional CNNs, especially in scenarios requiring the understanding of global visual contexts like medical imaging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。