提出GNNFormer，让图Transformer更抗噪声、更高效。
Rethinking Graph Transformer Architecture Design for Node Classification
- 拆分传播与变换模块，重构图注意力结构
- 12个数据集验证，抗噪声且计算效率高
- 适合同质与异质图分类任务
图Transformer（GT）作为图神经网络的一种，利用多头注意力实现高阶消息传递。然而，这在节点分类任务中带来若干局限：1）节点易受全局噪声影响；2）自注意力计算难以扩展至大规模图。本文通过大量实验探究了GT架构在节点分类中的适应性，得出结论：当前GT中的多头自注意力模块可被完全替代，而前馈神经网络模块具有重要价值。基于此，我们解耦图神经网络的传播（P）与变换（T）过程，提出一种新型GT架构GNNFormer，其基于P/T组合的消息传递机制，适用于同质与异质场景下的节点分类。在12个基准数据集上的广泛实验表明，所提架构能有效适应节点分类任务，不受全局噪声干扰且具备良好计算效率。
原文摘要 · Abstract (English)
Graph Transformer (GT), as a special type of Graph Neural Networks (GNNs), utilizes multi-head attention to facilitate high-order message passing. However, this also imposes several limitations in node classification applications: 1) nodes are susceptible to global noise; 2) self-attention computation cannot scale well to large graphs. In this work, we conduct extensive observational experiments to explore the adaptability of the GT architecture in node classification tasks and draw several conclusions: the current multi-head self-attention module in GT can be completely replaceable, while the feed-forward neural network module proves to be valuable. Based on this, we decouple the propagation (P) and transformation (T) of GNNs and explore a powerful GT architecture, named GNNFormer, which is based on the P/T combination message passing and adapted for node classification in both homophilous and heterophilous scenarios. Extensive experiments on 12 benchmark datasets demonstrate that our proposed GT architecture can effectively adapt to node classification tasks without being affected by global noise and computational efficiency limitations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。