arXiv:2411.07123cs.LGcs.AI2024-11被引 1

提出高效动态图节点表示学习框架,提升模型鲁棒性与速度。

Fast and Robust Contextual Node Representation Learning over Dynamic Graphs

  • 基于稀疏节点注意力和优化视角重设计PPR传播机制
  • 效率提升6倍,噪声环境下性能显著优于顶尖方法
  • 适合需要实时更新、抗干扰强的动态图应用

现实世界图随时间不断新增边和节点,亟需高效维护稳定的节点表示。现有高效图神经网络将消息传递与学习解耦,多采用个性化页面排名(PPR)作为特征传播机制。然而多数PPR-based GNN针对静态图设计,高效维护动态图中的PPR仍是开放问题。此外,尽管PPR表现优异,其理论依据却甚少。本文受最新将PPR表述为显式ℓ₁正则化优化问题的启发,提出统一的动态图学习框架,结合稀疏节点注意力。我们给出选择PPR的合理性质,并作为未来注意力设计的指导。利用等价优化形式,采用近端梯度法(ISTA),使基于PPR的GNN效率提升至6倍。最后,构建简单而有效的模型GoPPE,通过最大化先前用作注意力的PPR引入稳健位置编码。在初始节点属性含噪时,GoPPE性能显著超越最先进基线,验证了其有效性和鲁棒性。

原文摘要 · Abstract (English)

Real-world graphs grow rapidly with edge and vertex insertions over time, motivating the problem of efficiently maintaining robust node representation over evolving graphs. Recent efficient GNNs are designed to decouple recursive message passing from the learning process, and favor Personalized PageRank (PPR) as the underlying feature propagation mechanism. However, most PPR-based GNNs are designed for static graphs, and efficient PPR maintenance remains as an open problem. Further, there is surprisingly little theoretical justification for the choice of PPR, despite its impressive empirical performance. In this paper, we are inspired by the recent PPR formulation as an explicit $\ell_1$-regularized optimization problem and propose a unified dynamic graph learning framework based on sparse node-wise attention. We also present a set of desired properties to justify the choice of PPR in STOA GNNs, and serves as the guideline for future node attention designs. Meanwhile, we take advantage of the PPR-equivalent optimization formulation and employ the proximal gradient method (ISTA) to improve the efficiency of PPR-based GNNs upto 6 times. Finally, we instantiate a simple-yet-effective model (\textsc{GoPPE}) with robust positional encodings by maximizing PPR previously used as attention. The model performs comparably to or better than the STOA baselines and greatly outperforms when the initial node attributes are noisy during graph evolution, demonstrating the effectiveness and robustness of \textsc{GoPPE}.

动态图图神经网络注意力机制鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。