arXiv:2506.00431cs.LG2025-06KDD被引 9

TIDFormer通过捕捉时间与交互动态,提升动态图Transformer的性能与效率。

TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer

  • 基于日历时间分块和一阶邻居提取交互嵌入,高效建模动态图变化。
  • 在多个数据集上超越现有最优模型,且推理速度显著更快。
  • 适合研究动态图建模、需要高效Transformer架构的开发者参考。

由于自注意力机制在序列建模中捕捉依赖关系的能力,现有动态图神经网络多采用Transformer架构并结合不同编码设计来捕获动态图的时序演化。然而,这些基于Transformer的动态图神经网络在效果和效率上差异显著,凸显了在动态图上合理定义自注意力机制以及全面编码时空与交互动态的重要性。本文提出TIDFormer,一种能高效充分利用时间与交互动态的动态图Transformer。我们澄清并验证了所提自注意力机制的可解释性,解决了此前方法在动态图上自注意力定义不可解释的开放问题。为分别建模时间与交互动态,我们采用基于日历的时间分块信息,并仅通过采样的一阶邻居提取双分图与非双分图的有信息量交互嵌入。此外,通过简单分解捕捉历史交互模式潜在变化,联合建模时间与交互特征。我们在多个动态图数据集上进行了广泛实验,结果表明TIDFormer在多数数据集与设置下均优于当前最优模型,且相比先前基于Transformer的方法展现出显著的效率优势。

原文摘要 · Abstract (English)

Due to the proficiency of self-attention mechanisms (SAMs) in capturing dependencies in sequence modeling, several existing dynamic graph neural networks (DGNNs) utilize Transformer architectures with various encoding designs to capture sequential evolutions of dynamic graphs. However, the effectiveness and efficiency of these Transformer-based DGNNs vary significantly, highlighting the importance of properly defining the SAM on dynamic graphs and comprehensively encoding temporal and interactive dynamics without extra complex modules. In this work, we propose TIDFormer, a dynamic graph TransFormer that fully exploits Temporal and Interactive Dynamics in an efficient manner. We clarify and verify the interpretability of our proposed SAM, addressing the open problem of its uninterpretable definitions on dynamic graphs in previous works. To model the temporal and interactive dynamics, respectively, we utilize the calendar-based time partitioning information and extract informative interaction embeddings for both bipartite and non-bipartite graphs using merely the sampled first-order neighbors. In addition, we jointly model temporal and interactive features by capturing potential changes in historical interaction patterns through a simple decomposition. We conduct extensive experiments on several dynamic graph datasets to verify the effectiveness and efficiency of TIDFormer. The experimental results demonstrate that TIDFormer excels, outperforming state-of-the-art models across most datasets and experimental settings. Furthermore, TIDFormer exhibits significant efficiency advantages compared to previous Transformer-based methods.

动态图Transformer时序建模图神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。