用节点特征导数提升图神经网络表达能力,效果媲美顶级模型
On The Expressive Power of GNN Derivatives
- 通过高阶导数生成结构感知嵌入,再由第二层GNN处理
- 理论证明其表达能力达到WL同构判别层级,超越普通GNN
- 适合研究图神经网络表达性或需增强模型判别力的场景
尽管图神经网络(GNN)取得显著进展,其表达能力有限仍是根本挑战。现有研究提出了多种高表达力架构,形成了表达力递增的层级体系。与此同时,对节点特征导数的研究广泛应用于过度压缩、过平滑现象及可解释性等领域,但尚未被用于提升表达力。本文首次揭示导数可作为增强GNN表达力的自然途径。提出高阶导数图神经网络(HOD-GNN),通过基模型的高阶节点导数生成结构感知嵌入,并由第二层GNN端到端训练处理。理论上,该架构族表达力与WL同构判别层级一致。还揭示了HOD-GNN与子图GNN及常见结构编码方案的深层关联。为提升效率,设计了利用图稀疏性和并行性的消息传递算法以计算高阶导数。在多个主流图学习基准测试中,HOD-GNN表现优异。
原文摘要 · Abstract (English)
Despite significant advances in Graph Neural Networks (GNNs), their limited expressivity remains a fundamental challenge. Research on GNN expressivity has produced many expressive architectures, leading to architecture hierarchies with models of increasing expressive power. Separately, derivatives of GNNs with respect to node features have been widely studied in the context of the oversquashing and over-smoothing phenomena, GNN explainability, and more. To date, these derivatives remain unexplored as a means to enhance GNN expressivity. In this paper, we show that these derivatives provide a natural way to enhance the expressivity of GNNs. We introduce High-Order Derivative GNN (HOD-GNN), a novel method that enhances the expressivity of Message Passing Neural Networks (MPNNs) by leveraging high-order node derivatives of the base model. These derivatives generate expressive structure-aware node embeddings processed by a second GNN in an end-to-end trainable architecture. Theoretically, we show that the resulting architecture family's expressive power aligns with the WL hierarchy. We also draw deep connections between HOD-GNN, Subgraph GNNs, and popular structural encoding schemes. For computational efficiency, we develop a message-passing algorithm for computing high-order derivatives of MPNNs that exploits graph sparsity and parallelism. Evaluations on popular graph learning benchmarks demonstrate HOD-GNN's strong performance on popular graph learning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。