arXiv:2602.19622cs.LGcs.AI2026-02

提出新型图注意力机制,提升模型效率与跨场景泛化能力

VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attention

  • 用双码本重构节点特征与图结构,生成语义图码
  • 在图令牌级别计算注意力,速度比传统方法快数倍
  • 在分布外场景下表现优异,适合大规模图数据

图变压器在图表示学习中展现出强大能力,但现有方法面临两大挑战:(1) 多数模型存在指数级增长的计算复杂度,难以扩展到大规模图;(2) 基于节点级操作的注意力机制限制了模型灵活性,导致分布外(OOD)场景下泛化性能差。为此,我们提出VecFormer(向量量化图变压器),一种高效且高度可泛化的节点分类模型,尤其适用于OOD设置。VecFormer采用两阶段训练:第一阶段使用两个码本重建节点特征和图结构,以学习丰富的语义图码;第二阶段基于跨码本变换,在图令牌级别执行注意力,降低计算复杂度并增强泛化能力。在不同规模数据集上的大量实验表明,VecFormer在性能和速度上均优于现有图变压器。

原文摘要 · Abstract (English)

Graph Transformer has demonstrated impressive capabilities in the field of graph representation learning. However, existing approaches face two critical challenges: (1) most models suffer from exponentially increasing computational complexity, making it difficult to scale to large graphs; (2) attention mechanisms based on node-level operations limit the flexibility of the model and result in poor generalization performance in out-of-distribution (OOD) scenarios. To address these issues, we propose \textbf{VecFormer} (the \textbf{Vec}tor Quantized Graph Trans\textbf{former}), an efficient and highly generalizable model for node classification, particularly under OOD settings. VecFormer adopts a two-stage training paradigm. In the first stage, two codebooks are used to reconstruct the node features and the graph structure, aiming to learn the rich semantic \texttt{Graph Codes}. In the second stage, attention mechanisms are performed at the \texttt{Graph Token} level based on the transformed cross codebook, reducing computational complexity while enhancing the model's generalization capability. Extensive experiments on datasets of various sizes demonstrate that VecFormer outperforms the existing Graph Transformer in both performance and speed.

图神经网络注意力机制高效模型泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。