提出新方法解决图神经网络深度增加时的过度平滑问题。
Solving Oversmoothing in GNNs via Nonlocal Message Passing: Algebraic Smoothing and Depth Scalability
- 在后层归一化架构中引入代数平滑机制
- 支持高达256层的网络且性能更优
- 无需额外参数，适合深层图神经网络研究
层归一化（LN）的位置与过度平滑现象之间的关系尚未得到充分探讨。我们发现一个关键矛盾：前置LN架构可避免过度平滑，但面临深度诅咒；后置LN架构规避了深度诅咒，却存在过度平滑问题。为解决该问题，我们提出一种基于后置LN的新方法，通过诱导代数平滑来防止过度平滑，同时避免深度诅咒。在五个基准数据集上的实验证明，该方法支持高达256层的深层网络，显著提升性能，且无需增加额外参数。主要贡献包括：理论分析，揭示了LN动态对过度平滑和深度诅咒的影响；提出一种原理性解决方案，实现代数平滑且兼顾深度可扩展性；大量实验验证了该方法在深层GNN中的有效性。
原文摘要 · Abstract (English)
The relationship between Layer Normalization (LN) placement and the oversmoothing phenomenon remains underexplored. We identify a critical dilemma: Pre-LN architectures avoid oversmoothing but suffer from the curse of depth, while Post-LN architectures bypass the curse of depth but experience oversmoothing. To resolve this, we propose a new method based on Post-LN that induces algebraic smoothing, preventing oversmoothing without the curse of depth. Empirical results across five benchmarks demonstrate that our approach supports deeper networks (up to 256 layers) and improves performance, requiring no additional parameters. Key contributions: Theoretical Characterization: Analysis of LN dynamics and their impact on oversmoothing and the curse of depth. A Principled Solution: A parameter-efficient method that induces algebraic smoothing and avoids oversmoothing and the curse of depth. Empirical Validation: Extensive experiments showing the effectiveness of the method in deeper GNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。