arXiv:2411.10000cs.LGcs.AI2024-11被引 7

提出双阶等变图微分方程,提升图神经网络对动态系统的建模能力。

DuSEGO: Dual Second-order Equivariant Graph Ordinary Differential Equation

  • 同时在节点嵌入与坐标上应用双阶等变图ODE,增强表达能力。
  • 理论证明可缓解过平滑与梯度消失/爆炸问题,支持深层网络训练。
  • 适用于分子性质预测等复杂动态系统建模,适合追求高精度的科研人员。

具有等变特性的图神经网络在建模复杂动态系统和分子性质方面已取得显著成果。然而,其表达能力受限于:(1) 现有方法常忽视传统GNN模型引起的过平滑问题,以及深层GNN中的梯度爆炸或消失问题;(2) 多数模型仅基于一阶信息,而现实世界多为二阶系统,限制了模型表征能力。为此,我们提出 extbf{Du}al extbf{S}econd-order extbf{E}quivariant extbf{G}raph extbf{O}rdinary Differential Equation( exttt{DuSEGO})用于等变表示学习。具体而言, exttt{DuSEGO} 在图嵌入和节点坐标上同时应用双阶等变图微分方程(Graph ODEs)。理论上,我们首先证明了 exttt{DuSEGO} 保持等变性;进一步,从理论层面揭示其有效缓解了特征表示与坐标更新中的过平滑问题。此外,实验表明该方法能有效缓解梯度爆炸与消失问题,有助于深层多层GNN的训练。在基准数据集上的大量实验验证了 exttt{DuSEGO} 相较于基线方法的优越性。

原文摘要 · Abstract (English)

Graph Neural Networks (GNNs) with equivariant properties have achieved significant success in modeling complex dynamic systems and molecular properties. However, their expressiveness ability is limited by: (1) Existing methods often overlook the over-smoothing issue caused by traditional GNN models, as well as the gradient explosion or vanishing problems in deep GNNs. (2) Most models operate on first-order information, neglecting that the real world often consists of second-order systems, which further limits the model's representation capabilities. To address these issues, we propose the \textbf{Du}al \textbf{S}econd-order \textbf{E}quivariant \textbf{G}raph \textbf{O}rdinary Differential Equation (\method{}) for equivariant representation. Specifically, \method{} apply the dual second-order equivariant graph ordinary differential equations (Graph ODEs) on graph embeddings and node coordinates, simultaneously. Theoretically, we first prove that \method{} maintains the equivariant property. Furthermore, we provide theoretical insights showing that \method{} effectively alleviates the over-smoothing problem in both feature representation and coordinate update. Additionally, we demonstrate that the proposed \method{} mitigates the exploding and vanishing gradients problem, facilitating the training of deep multi-layer GNNs. Extensive experiments on benchmark datasets validate the superiority of the proposed \method{} compared to baselines.

图神经网络等变模型微分方程分子建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。