arXiv:2502.08101cs.LGcs.AI2025-02NeurIPS被引 7

通过节点互换生成更丰富序列,提升图注意力模型分类性能

Rethinking Tokenized Graph Transformers for Node Classification

  • 引入节点互换机制,利用语义相关性生成更丰富的令牌序列
  • 在多个数据集上实现优于现有方法的节点分类准确率
  • 适合需要提升图神经网络表达能力的研究者参考

节点令牌化图变换器(GT)在节点分类任务中展现出良好性能。现有方法的核心是将输入图转换为令牌序列,通过Transformer学习节点表示。本文发现,现有方法仅关注构造相似图中的一阶邻域节点,导致生成的令牌序列多样性不足,限制了模型潜力。为此,提出SwapGT方法:首先基于令牌序列特性设计新的节点互换操作,充分挖掘节点间的语义相关性以生成更丰富的令牌序列;其次采用Transformer骨干网络从生成的序列中学习节点表示;此外,设计中心对齐损失约束多序列表示学习,进一步提升性能。大量实验证明,SwapGT在多个数据集上均优于现有方法。

原文摘要 · Abstract (English)

Node tokenized graph Transformers (GTs) have shown promising performance in node classification. The generation of token sequences is the key module in existing tokenized GTs which transforms the input graph into token sequences, facilitating the node representation learning via Transformer. In this paper, we observe that the generations of token sequences in existing GTs only focus on the first-order neighbors on the constructed similarity graphs, which leads to the limited usage of nodes to generate diverse token sequences, further restricting the potential of tokenized GTs for node classification. To this end, we propose a new method termed SwapGT. SwapGT first introduces a novel token swapping operation based on the characteristics of token sequences that fully leverages the semantic relevance of nodes to generate more informative token sequences. Then, SwapGT leverages a Transformer-based backbone to learn node representations from the generated token sequences. Moreover, SwapGT develops a center alignment loss to constrain the representation learning from multiple token sequences, further enhancing the model performance. Extensive empirical results on various datasets showcase the superiority of SwapGT for node classification.

图神经网络注意力机制节点分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。