残差连接可有效缓解图神经网络深度下的特征混淆问题。
Residual connections provably mitigate oversmoothing in graph neural networks
- 基于乘法遍历定理，推导出节点相似度的渐近收敛速率。
- 证明残差连接能显著减缓多类参数分布下的过平滑现象。
- 理论与实验结合，适用于各类深度图神经网络设计参考。
图神经网络（GNN）在处理图结构数据方面取得了显著的实证成功，但深层网络中普遍存在的“过平滑”问题仍制约其表达能力——节点特征趋于相似，失去区分性。本文通过乘法遍历定理，推导了含与不含残差连接的深层GNN的节点相似度渐近收敛速率。结果表明，残差连接可有效缓解或抑制多种参数分布下的过平滑现象。理论分析得到数值实验的有力支持。
原文摘要 · Abstract (English)
Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。