通过图像分块构建高效视觉图网络,显著降低推理延迟。
ClusterViG: Efficient Globally Aware Vision GNNs via Image Partitioning
- 将图像分块并行构建图结构,提升图构造效率
- 支持局部与全局特征学习,增强全局上下文感知能力
- 适合高分辨率图像训练,适用于分类检测等任务
卷积神经网络(CNN)和视觉变压器(ViT)在计算机视觉领域占据主导地位。图神经网络(GNN)因能通过非结构化图表示复杂关系而在多个领域表现优异。然而,视觉图网络(ViG)的适用性直到最近才被探索。尽管ViG取得成功,其性能严重受限于基于k近邻(k-NN)的图构建开销。近期工作虽缓解该瓶颈,但限制了GNN构建非结构化图的灵活性,削弱其核心优势并引入额外效率损失。为此,本文提出动态高效图卷积(DEGC),用于设计高效且具备全局感知能力的视觉图网络。DEGC将输入图像分块,并为每块并行构建图,提升图构造效率;同时融合局部图内与全局图间特征学习,增强全局上下文感知。以DEGC为基础,构建新型混合架构ClusterViG。大量实验表明,相较于ViG、ViHGNN、PVG和GreedyViG等模型,ClusterViG在端到端推理延迟上最多降低5倍,参数量相近。此外,ClusterViG在图像分类、目标检测和实例分割任务上达到当前最优性能,验证了全局感知学习策略的有效性。最后,DEGC的图像分块机制使ClusterViG可高效训练更高分辨率图像,体现方法可扩展性。
原文摘要 · Abstract (English)
Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have dominated the field of Computer Vision (CV). Graph Neural Networks (GNN) have performed remarkably well across diverse domains because they can represent complex relationships via unstructured graphs. However, the applicability of GNNs for visual tasks was unexplored till the introduction of Vision GNNs (ViG). Despite the success of ViGs, their performance is severely bottlenecked due to the expensive $k$-Nearest Neighbors ($k$-NN) based graph construction. Recent works addressing this bottleneck impose constraints on the flexibility of GNNs to build unstructured graphs, undermining their core advantage while introducing additional inefficiencies. To address these issues, in this paper, we propose a novel method called Dynamic Efficient Graph Convolution (DEGC) for designing efficient and globally aware ViGs. DEGC partitions the input image and constructs graphs in parallel for each partition, improving graph construction efficiency. Further, DEGC integrates local intra-graph and global inter-graph feature learning, enabling enhanced global context awareness. Using DEGC as a building block, we propose a novel CNN-GNN architecture, ClusterViG, for CV tasks. Extensive experiments indicate that ClusterViG reduces end-to-end inference latency for vision tasks by up to $5\times$ when compared against a suite of models such as ViG, ViHGNN, PVG, and GreedyViG, with a similar model parameter count. Additionally, ClusterViG reaches state-of-the-art performance on image classification, object detection, and instance segmentation tasks, demonstrating the effectiveness of the proposed globally aware learning strategy. Finally, input partitioning performed by DEGC enables ClusterViG to be trained efficiently on higher-resolution images, underscoring the scalability of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。