改进视觉Transformer的注意力机制,提升目标检测精度。
Unified Local and Global Attention Interaction Modeling for Vision Transformers
- 引入局部卷积池化增强相邻特征交互
- 通过概念注意力转换促进语义间信息交换
- 在多个医学数据集上表现优异,适合医疗图像分析
我们提出一种新方法,扩展视觉变换器(ViT)的自注意力机制,以在多种数据集上实现更精准的目标检测。ViT在图像理解任务中表现出色,部分原因在于其能利用视觉标记间的全局信息交互。然而,传统ViT的自注意力机制存在局限:在计算全局注意力前,视觉标记无法与邻近特征交换局部或全局信息,导致标记被孤立处理,空间关系被忽略。此外,点积相似度操作使不同语义类别的标记外观相似,加剧了混淆。为此,我们对自注意力框架进行了两项改进:一是采用新型激进卷积池化策略实现局部特征混合;二是引入概念注意力转换,促进语义概念间的交互与特征交换。实验表明,自注意力前进行局部与全局信息交换显著提升了复杂目标检测任务的表现,并在多个基准数据集及挑战性医学数据集上具备良好泛化能力。我们开源了源代码和一个包含癌变肿瘤的新数据集(chimeric cell clusters)。
原文摘要 · Abstract (English)
We present a novel method that extends the self-attention mechanism of a vision transformer (ViT) for more accurate object detection across diverse datasets. ViTs show strong capability for image understanding tasks such as object detection, segmentation, and classification. This is due in part to their ability to leverage global information from interactions among visual tokens. However, the self-attention mechanism in ViTs are limited because they do not allow visual tokens to exchange local or global information with neighboring features before computing global attention. This is problematic because tokens are treated in isolation when attending (matching) to other tokens, and valuable spatial relationships are overlooked. This isolation is further compounded by dot-product similarity operations that make tokens from different semantic classes appear visually similar. To address these limitations, we introduce two modifications to the traditional self-attention framework; a novel aggressive convolution pooling strategy for local feature mixing, and a new conceptual attention transformation to facilitate interaction and feature exchange between semantic concepts. Experimental results demonstrate that local and global information exchange among visual features before self-attention significantly improves performance on challenging object detection tasks and generalizes across multiple benchmark datasets and challenging medical datasets. We publish source code and a novel dataset of cancerous tumors (chimeric cell clusters).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。