提出可学习的图构建方法,让视觉图网络自动找最佳邻居关系。
ViG-LRGC: Vision Graph Neural Networks with Learnable Reparameterized Graph Construction
- 用键值注意力加软阈值重参数化,实现可训练的图边选择。
- 在ImageNet-1k上性能超越同类先进模型,无需超参调优。
- 适合追求自适应图结构与端到端训练的视觉模型研究者。
图像表示学习是计算机视觉的重要问题。传统方法将图像处理为网格(如卷积神经网络)或视觉标记序列(如Vision Transformer)。近年来,视觉图神经网络(ViG)将图像视为节点图,提供更直观的表示。但如何在每一层构建最优节点关系图且无需超参搜索仍具挑战。现有ViG模型依赖非参数化、不可学习的统计方法构建图,可能无法为每个节点选出最佳邻域。从k-NN到超图、相似度阈值等方法均缺乏可学习且免超参的图构建机制。为此,本文提出可学习重参数化图构建(LRGC),对每对节点应用键值注意力,再通过软阈值重参数化进行边选择,实现可微训练。利用可学习参数选择邻域,消除聚类或阈值方法带来的偏差。同时,阈值可在各层随训练数据自适应调整,无需人工设定。实验表明,所提的ViG-LRGC在ImageNet-1k上优于同等规模的先进ViG模型。
原文摘要 · Abstract (English)
Image Representation Learning is an important problem in Computer Vision. Traditionally, images were processed as grids, using Convolutional Neural Networks or as a sequence of visual tokens, using Vision Transformers. Recently, Vision Graph Neural Networks (ViG) have proposed the treatment of images as a graph of nodes; which provides a more intuitive image representation. The challenge is to construct a graph of nodes in each layer that best represents the relations between nodes and does not need a hyper-parameter search. ViG models in the literature depend on non-parameterized and non-learnable statistical methods that operate on the latent features of nodes to create a graph. This might not select the best neighborhood for each node. Starting from k-NN graph construction to HyperGraph Construction and Similarity-Thresholded graph construction, these methods lack the ability to provide a learnable hyper-parameter-free graph construction method. To overcome those challenges, we present the Learnable Reparameterized Graph Construction (LRGC) for Vision Graph Neural Networks. LRGC applies key-query attention between every pair of nodes; then uses soft-threshold reparameterization for edge selection, which allows the use of a differentiable mathematical model for training. Using learnable parameters to select the neighborhood removes the bias that is induced by any clustering or thresholding methods previously introduced in the literature. In addition, LRGC allows tuning the threshold in each layer to the training data since the thresholds are learnable through training and are not provided as hyper-parameters to the model. We demonstrate that the proposed ViG-LRGC approach outperforms state-of-the-art ViG models of similar sizes on the ImageNet-1k benchmark dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。