用交叉注意力动态聚合视觉图神经网络邻域信息,性能超越同类模型。
AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs
- 以节点为查询、邻居为键,通过交叉注意力捕捉复杂邻域关系。
- 在ImageNet-1K上达到当前最优,且在多个下游任务中表现优异。
- 无需特定结构优化,兼具高精度与计算效率,适合通用视觉任务。
视觉图神经网络(ViGs)在图像识别任务中展现出优于卷积神经网络(CNNs)和视觉变压器(ViTs)的潜力。其核心在于节点与邻域特征的聚合方法。尽管已有多种图卷积方法(如Max-Relative、EdgeConv、GIN、GraphSAGE)被探索,但仍缺乏一种无需架构定制即可有效建模复杂节点-邻域关系的通用聚合方法。为此,本文提出一种基于交叉注意力的聚合机制:查询投影来自节点自身,键投影来自其邻居。进一步设计了名为AttentionViG的新架构,采用该交叉注意力聚合方案实现非局部消息传递。在ImageNet-1K基准上评估时,AttentionViG达到当前最优性能。同时,在MS COCO 2017的目标检测与实例分割任务,以及ADE20K语义分割任务上验证其迁移能力。结果表明,该方法不仅性能强劲,还保持高效,在计算量(FLOPs)与先前视觉图神经网络相当的前提下,实现竞争力准确率。
原文摘要 · Abstract (English)
Vision Graph Neural Networks (ViGs) have demonstrated promising performance in image recognition tasks against Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). An essential part of the ViG framework is the node-neighbor feature aggregation method. Although various graph convolution methods, such as Max-Relative, EdgeConv, GIN, and GraphSAGE, have been explored, a versatile aggregation method that effectively captures complex node-neighbor relationships without requiring architecture-specific refinements is needed. To address this gap, we propose a cross-attention-based aggregation method in which the query projections come from the node, while the key projections come from its neighbors. Additionally, we introduce a novel architecture called AttentionViG that uses the proposed cross-attention aggregation scheme to conduct non-local message passing. We evaluated the image recognition performance of AttentionViG on the ImageNet-1K benchmark, where it achieved SOTA performance. Additionally, we assessed its transferability to downstream tasks, including object detection and instance segmentation on MS COCO 2017, as well as semantic segmentation on ADE20K. Our results demonstrate that the proposed method not only achieves strong performance, but also maintains efficiency, delivering competitive accuracy with comparable FLOPs to prior vision GNN architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。