arXiv:2505.17660cs.LG2025-05被引 2

解决图神经网络中邻居信息混淆问题,提升节点分类精度。

DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification

  • 用双重位置编码融合拓扑与属性相关性,更好捕捉邻居特征。
  • 引入注意力掩码机制,防止远距离邻居干扰关键信息交互。
  • 在多类图数据上表现优于现有方法,适合复杂图结构分析。

基于邻域感知的图变换器在节点分类任务中展现出巨大潜力。然而,我们对邻域标记的深入分析揭示了现有范式的两个关键局限:其一,当前邻域标记生成方法未能充分捕捉邻域内的属性相关性;其二,传统自注意力机制在处理邻域标记时存在注意力分散问题,高跳数邻域获得过度关注,严重破坏目标节点与其邻域标记之间的信息交互。为此,我们提出DAM-GT——基于双位置编码的注意力掩码图变压器。DAM-GT引入一种新颖的双位置编码方案,通过属性聚类策略实现属性感知编码,有效保留了节点在拓扑和属性空间中的相关性。同时,DAM-GT设计了一种新的注意力机制,采用简单而有效的掩码策略,引导目标节点与其邻域标记之间的交互,克服注意力分散问题。在不同同质性水平和规模的多种图数据上的大量实验表明,DAM-GT在节点分类任务中持续优于现有最先进方法。

原文摘要 · Abstract (English)

Neighborhood-aware tokenized graph Transformers have recently shown great potential for node classification tasks. Despite their effectiveness, our in-depth analysis of neighborhood tokens reveals two critical limitations in the existing paradigm. First, current neighborhood token generation methods fail to adequately capture attribute correlations within a neighborhood. Second, the conventional self-attention mechanism suffers from attention diversion when processing neighborhood tokens, where high-hop neighborhoods receive disproportionate focus, severely disrupting information interactions between the target node and its neighborhood tokens. To address these challenges, we propose DAM-GT, Dual positional encoding-based Attention Masking graph Transformer. DAM-GT introduces a novel dual positional encoding scheme that incorporates attribute-aware encoding via an attribute clustering strategy, effectively preserving node correlations in both topological and attribute spaces. In addition, DAM-GT formulates a new attention mechanism with a simple yet effective masking strategy to guide interactions between target nodes and their neighborhood tokens, overcoming the issue of attention diversion. Extensive experiments on various graphs with different homophily levels as well as different scales demonstrate that DAM-GT consistently outperforms state-of-the-art methods in node classification tasks.

图神经网络注意力机制节点分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。