用序列化图令牌提升图注意力模型的全局表征能力
Enhanced Graph Transformer with Serialized Graph Tokens
- 将图节点序列化为带位置编码的图令牌，增强自注意力建模
- 在多个图级任务上达到当前最佳性能，超越基线方法
- 适合需要强全局表征的图神经网络研究者使用
Transformer 在图学习中已取得成功，尤其在节点级任务表现突出。然而，现有方法在生成图级表示时面临信息瓶颈。主流的单令牌范式无法充分挖掘自注意力对序列的建模优势，退化为节点信号的加权求和。为此，我们提出一种新型序列化令牌范式，更有效地封装全局信号。具体地，设计图序列化方法将节点信号聚合为序列化的图令牌，并自动引入位置编码；随后通过堆叠的自注意力层编码该令牌序列，捕捉其内部依赖关系。该方法通过建模多个图令牌间的复杂交互，生成更具表达力的图表示。实验表明，该方法在多个图级基准上达到最先进水平。消融实验证明了所提模块的有效性。
原文摘要 · Abstract (English)
Transformers have demonstrated success in graph learning, particularly for node-level tasks. However, existing methods encounter an information bottleneck when generating graph-level representations. The prevalent single token paradigm fails to fully leverage the inherent strength of self-attention in encoding token sequences, and degenerates into a weighted sum of node signals. To address this issue, we design a novel serialized token paradigm to encapsulate global signals more effectively. Specifically, a graph serialization method is proposed to aggregate node signals into serialized graph tokens, with positional encoding being automatically involved. Then, stacked self-attention layers are applied to encode this token sequence and capture its internal dependencies. Our method can yield more expressive graph representations by modeling complex interactions among multiple graph tokens. Experimental results show that our method achieves state-of-the-art results on several graph-level benchmarks. Ablation studies verify the effectiveness of the proposed modules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。