arXiv:2509.14863cs.LGcs.AI2025-09中稿 · Frontiers of Compu…

提出新型全局到局部注意力机制,提升图变换器对局部结构的感知能力。

Exploring the Global-to-Local Attention Scheme in Graph Transformers: An Empirical Study

  • 先用注意力学全局信息,再用GNN学局部结构,避免信息丢失。
  • 在节点和图级别任务上均达到顶尖性能,且复杂度保持线性。
  • 适合关注图结构建模与注意力机制融合的研究者。

图变换器(GTs)在图表示学习中展现出巨大潜力。其架构通常将图神经网络(GNN)与全局注意力机制并行或顺序结合,形成局部-全局或局部到全局的注意力方案。然而,由于全局注意力主要捕捉节点间的长程依赖,这类整合方式可能导致由GNN学习的局部邻域信息被稀释。为此,我们提出G2LFormer,采用新颖的全局到局部注意力机制:浅层使用注意力捕捉全局信息,深层则通过GNN模块学习局部结构信息,从而防止节点忽略其直接邻居。同时引入有效的跨层信息融合策略,使局部层可保留全局层的有益信息,缓解信息损失,且在可接受的可扩展性代价下实现。为验证全局到局部注意力方案的有效性,我们在节点级和图级任务上对比了G2LFormer与最先进的线性图变换器及GNN模型。结果表明,G2LFormer在保持线性复杂度的同时表现出卓越性能。

原文摘要 · Abstract (English)

Graph Transformers (GTs) show considerable potential in graph representation learning. The architecture of GTs typically integrates Graph Neural Networks (GNNs) with global attention mechanisms either in parallel or as a precursor to attention mechanisms, yielding a local-and-global or local-to-global attention scheme. However, as the global attention mechanism primarily captures long-range dependencies between nodes, these integration schemes may suffer from information loss, where the local neighborhood information learned by GNN could be diluted by the attention mechanism. Therefore, we propose G2LFormer, featuring a novel global-to-local attention scheme where the shallow network layers use attention mechanisms to capture global information, while the deeper layers employ GNN modules to learn local structural information, thereby preventing nodes from ignoring their immediate neighbors. An effective cross-layer information fusion strategy is introduced to allow local layers to retain beneficial information from global layers and alleviate information loss, with acceptable trade-offs in scalability. To validate the feasibility of the global-to-local attention scheme, we compare G2LFormer with state-of-the-art linear GTs and GNNs on node-level and graph-level tasks. The results indicate that G2LFormer exhibits excellent performance while keeping linear complexity.

图神经网络注意力机制结构学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。