通过信号传播分析优化图神经网络初始化，让深层模型不再变差。
Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective
- 从信号前向、反向传播和图嵌入变化三方面定义深层GNN的初始化标准
- 提出SPoGInit方法，使深度增加时性能不降反升
- 适合研究深层图神经网络或需要稳定训练的工程师
图神经网络（GNN）在深度增加时常出现性能下降问题。本文从信号传播（SP）视角出发，提出三个衡量有效信号传播的关键指标：前向传播、反向传播和图嵌入变化（GEV）。前两个指标源自经典信号传播理论，第三个为针对GNN设计的新指标。理论上证明，现有广泛使用的初始化方法无法同时控制这三个指标，导致深度增加时性能下降。为此，本文基于信号传播分析，直接搜索最优权重初始化方差以优化三指标，提出图引导的信号传播初始化方法（SPoGInit）。实验表明，SPoGInit在多种任务和架构上均优于常见初始化方法；更重要的是，它使深层GCN性能随深度增加而提升，显著缓解了深度相关挑战，验证了信号传播分析框架的有效性。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) often suffer from performance degradation as the network depth increases. This paper addresses this issue by introducing initialization methods that enhance signal propagation (SP) within GNNs. We propose three key metrics for effective SP in GNNs: forward propagation, backward propagation, and graph embedding variation (GEV). While the first two metrics derive from classical SP theory, the third is specifically designed for GNNs. We theoretically demonstrate that a broad range of commonly used initialization methods for GNNs, which exhibit performance degradation with increasing depth, fail to control these three metrics simultaneously. To deal with this limitation, a direct exploitation of the SP analysis--searching for weight initialization variances that optimize the three metrics--is shown to significantly enhance the SP in deep GCNs. This approach is called Signal Propagation on Graph-guided Initialization (SPoGInit). Our experiments demonstrate that SPoGInit outperforms commonly used initialization methods on various tasks and architectures. Notably, SPoGInit enables performance improvements as GNNs deepen, which represents a significant advancement in addressing depth-related challenges and highlights the validity and effectiveness of the SP analysis framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。