用聚类注意力扩大图神经网络感知范围，兼顾结构信息与全局视野。
Cluster Attention for Graph Machine Learning
- 将节点分簇后让每点关注同簇所有节点，提升感受野。
- 在多个图数据集上显著提升MPNN和Graph Transformer性能。
- 适合需要结构先验的图学习任务，如社交网络分析。
消息传递神经网络（MPNN）是当前图机器学习主流方法，但其感受野受限于消息传递层数。为扩大感受野，已有研究提出具有全局注意力的图变压器（Graph Transformers），但全局注意力忽略图拓扑结构，缺乏图结构相关的归纳偏置，而这类偏置对图学习任务至关重要。本文提出一种新方法：聚类注意力（Cluster Attention, CLATT）。利用现成的图社区检测算法将节点划分为若干簇，并让每个节点关注所属簇内的所有其他节点。该方法在保持强图结构归纳偏置的同时实现大感受野。实验表明，将CLATT引入MPNN或图变压器，在涵盖图学习新基准GraphLand的真实应用数据集上的多种任务中均显著提升性能。
原文摘要 · Abstract (English)
Message Passing Neural Networks have recently become the most popular approach to graph machine learning tasks; however, their receptive field is limited by the number of message passing layers. To increase the receptive field, Graph Transformers with global attention have been proposed; however, global attention does not take into account the graph topology and thus lacks graph-structure-based inductive biases, which are typically very important for graph machine learning tasks. In this work, we propose an alternative approach: cluster attention (CLATT). We divide graph nodes into clusters with off-the-shelf graph community detection algorithms and let each node attend to all other nodes in each cluster. CLATT provides large receptive fields while still having strong graph-structure-based inductive biases. We show that augmenting Message Passing Neural Networks or Graph Transformers with CLATT significantly improves their performance on a wide range of graph datasets including datasets from the recently introduced GraphLand benchmark representing real-world applications of graph machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。