提出深度谱扩散模型,解决深层图神经网络信号衰减问题。
Deep Neural Sheaf Diffusion

- 用谱邻接算子替代拉普拉斯算子,保持深层信息传递
- 在合成数据上准确率提升30个百分点,真实数据持续领先
- 适合构建深度图神经网络与图基础模型的研究者
深度图神经网络(GNN)对捕捉图结构数据中的复杂依赖关系至关重要,但其深度扩展面临挑战:层叠导致表示坍缩和敏感性下降。尽管神经谱扩散(NSD)具备理论上的抗坍缩保障,但在实践中随深度增加,谱拉普拉斯的分歧信号消失,限制深层贡献。本文识别出阻碍NSD在深层有效性的机制,提出深度神经谱扩散(DNSD),将谱拉普拉斯替换为谱邻接算子,以维持各层间的信息信号。该方法结合归一化、奇数非线性与门控机制。通过对比谱扩散与图注意力机制,揭示DNSD以矩阵值边函数取代标量注意力分数,并对节点表示而非注意力分数进行归一化。实验表明,DNSD在图任务中有效利用深层聚合,在合成长程数据集上准确率最高提升30个百分点,在真实世界基准上也持续优于基线模型。这使基于谱的架构成为构建图基础模型的有力候选。
原文摘要 · Abstract (English)
Deep Graph Neural Networks (GNNs) are essential for capturing complex dependencies in graph-structured data. However, scaling GNNs to depth remains challenging, as stacking layers leads to representation collapse and diminishing sensitivity due to repeated aggregation. While Neural Sheaf Diffusion (NSD) provides strong theoretical guarantees against such collapse, these guarantees do not translate to practice: as depth increases, the disagreement signal of the sheaf Laplacian vanishes, limiting the contribution of deeper layers. We identify mechanisms that hinder NSD effectiveness at depth and propose \emph{Deep Neural Sheaf Diffusion} (DNSD), which replaces the sheaf Laplacian with a sheaf adjacency operator to maintain informative signals across layers. This is complemented by normalization, odd nonlinearities, and gating. To provide a principled explanation of the expected performance improvement, we contrast sheaf diffusion to graph attention mechanisms, highlighting that DNSD replaces scalar attention scores with matrix-valued edge functions and normalizes node representations rather than attention scores. We demonstrate empirically that DNSD effectively utilizes deep aggregation in graph tasks, outperforming GNN and NSD baselines with up to 30pp accuracy on synthetic long-range datasets, and consistently outperforming them on real-world benchmarks. These results position sheaf-based architectures as a promising building block for graph foundation models by supporting effective deep architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。