arXiv:2412.18221cs.CVcs.LG2024-12被引 2

用自适应图结构和GNN提升图像匹配精度,效果显著优于传统方法。

GIMS: Image Matching System Based on Adaptive Graph Construction and Graph Neural Network

  • 基于距离与动态阈值的自适应图构建,减少冗余连接。
  • 融合GNN与Transformer,提升空间与特征表示能力,匹配性能平均提升3.8到40.3倍。
  • 适用于高精度图像匹配任务,尤其适合需要高效训练的场景。

基于特征的图像匹配在计算机视觉中应用广泛。图像中的关键点可自然表示为图结构,而图神经网络(GNN)已被证明优于传统深度学习方法。本文提出一种创新的自适应图构造方法,基于距离与动态阈值相似性进行筛选,根据已有节点特性动态调整新节点的加入标准,构建更精确且鲁棒的图结构。进一步结合GNN的顶点处理能力与Transformer的全局感知能力,增强图结构中空间与特征信息的表达,深化对顶点间关系及其匹配贡献的理解。采用Sinkhorn算法迭代求解最优匹配结果。在多个图像数据集上验证系统性能,对比实验表明,本系统整体匹配性能平均提升3.8倍至40.3倍。同时,顶点数与边数显著影响训练效率与内存消耗,因此采用多GPU技术加速训练过程。代码已开源:https://github.com/songxf1024/GIMS。

原文摘要 · Abstract (English)

Feature-based image matching has extensive applications in computer vision. Keypoints detected in images can be naturally represented as graph structures, and Graph Neural Networks (GNNs) have been shown to outperform traditional deep learning techniques. Consequently, the paradigm of image matching via GNNs has gained significant prominence in recent academic research. In this paper, we first introduce an innovative adaptive graph construction method that utilizes a filtering mechanism based on distance and dynamic threshold similarity. This method dynamically adjusts the criteria for incorporating new vertices based on the characteristics of existing vertices, allowing for the construction of more precise and robust graph structures while avoiding redundancy. We further combine the vertex processing capabilities of GNNs with the global awareness capabilities of Transformers to enhance the model's representation of spatial and feature information within graph structures. This hybrid model provides a deeper understanding of the interrelationships between vertices and their contributions to the matching process. Additionally, we employ the Sinkhorn algorithm to iteratively solve for optimal matching results. Finally, we validate our system using extensive image datasets and conduct comprehensive comparative experiments. Experimental results demonstrate that our system achieves an average improvement of 3.8x-40.3x in overall matching performance. Additionally, the number of vertices and edges significantly impacts training efficiency and memory usage; therefore, we employ multi-GPU technology to accelerate the training process. Our code is available at https://github.com/songxf1024/GIMS.

图像匹配图神经网络自适应图深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。