为动态图设计新编码，让Transformer更好捕捉时空信息。
Supra-Laplacian Encoding for Transformer on Dynamic Graphs
- 将动态图转为多层图，利用超拉普拉斯矩阵的谱特性编码时空结构。
- 引入交叉注意力建模节点对关系，提升动态链接预测精度。
- 在9个数据集上超越主流方法，适合动态图建模任务。
全连接图Transformer（GT）在静态图中表现优异，但面对动态图时，因在多个时间快照间连接所有节点，导致失去结构与时间信息。本文提出时空变换器（SLATE），通过将离散时间动态图转化为多层图，并利用其关联的超拉普拉斯矩阵的谱特性进行编码，保留时空信息。第二项贡献是引入交叉注意力机制，显式建模节点间配对关系，提供精确的边表示，用于动态链接预测。SLATE在9个数据集上优于多种基于消息传递图神经网络与循环模型（如LSTM）或动态图Transformer的先进方法。代码已开源：github.com/ykrmm/SLATE。
原文摘要 · Abstract (English)
Fully connected Graph Transformers (GT) have rapidly become prominent in the static graph community as an alternative to Message-Passing models, which suffer from a lack of expressivity, oversquashing, and under-reaching. However, in a dynamic context, by interconnecting all nodes at multiple snapshots with self-attention, GT loose both structural and temporal information. In this work, we introduce Supra-LAplacian encoding for spatio-temporal TransformErs (SLATE), a new spatio-temporal encoding to leverage the GT architecture while keeping spatio-temporal information. Specifically, we transform Discrete Time Dynamic Graphs into multi-layer graphs and take advantage of the spectral properties of their associated supra-Laplacian matrix. Our second contribution explicitly model nodes' pairwise relationships with a cross-attention mechanism, providing an accurate edge representation for dynamic link prediction. SLATE outperforms numerous state-of-the-art methods based on Message-Passing Graph Neural Networks combined with recurrent models (e.g LSTM), and Dynamic Graph Transformers, on 9 datasets. Code is available at: github.com/ykrmm/SLATE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。