arXiv:2607.28185cs.LGcs.AI2026-07

用持续高斯噪声防止图神经网络表示过度平滑

Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

  • 每步传播后注入独立高斯噪声,构建随机动力系统
  • 理论证明噪声能保持节点表示多样性,避免能量趋零
  • 适合研究深层图神经网络稳定性的研究人员

深度图神经网络中的过度平滑问题源于重复消息传递使节点表示趋于相似,最终坍缩至低维子空间,限制了模型的有效深度。本文研究一种在每次传播后注入独立高斯噪声的循环图神经网络,并将其建模为随机动力系统。在标准全局收缩假设下,证明隐藏表示构成几何遍历的马尔可夫链,存在唯一不变概率测度。主要理论结果表明,平稳状态下的期望狄利克雷能量具有显式的正下界,与噪声方差和图的谱隙成正比。因此,平稳表示不会坍缩到常数流形,严格保证了渐近过度平滑被抑制(以狄利克雷能量不消失为标志)。分析揭示持久随机扰动是一种区别于残差连接、归一化和图重连等确定性方法的根本新机制。数值实验在线性和非线性循环GNN上均与理论预测高度一致,验证了平稳分布的存在及极限狄利克雷能量对噪声强度的依赖关系。

原文摘要 · Abstract (English)

Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace. This phenomenon limits the effective depth of message-passing architectures and motivates the search for mechanisms that preserve representation diversity. In this paper, we study a recurrent graph neural network in which independent Gaussian noise is injected after every propagation step and analyze the resulting architecture as a stochastic dynamical system. Under a standard global contraction assumption on the deterministic update, we prove that the hidden representations form a geometrically ergodic Markov chain admitting a unique invariant probability measure. Our main theoretical result establishes an explicit positive lower bound on the expected stationary Dirichlet energy, proportional to both the noise variance and the spectral gap of the underlying graph. Consequently, the stationary representations cannot collapse onto the constant manifold, providing a rigorous guarantee that asymptotic oversmoothing is prevented in the sense of non-vanishing Dirichlet energy. Our analysis reveals persistent stochastic perturbations as a fundamentally different mechanism for combating oversmoothing, complementing existing deterministic approaches based on residual connections, normalization, and graph rewiring. Finally, numerical experiments on both linear and nonlinear recurrent graph neural networks closely match the theoretical predictions, illustrating the emergence of a stationary distribution and the predicted dependence of the limiting Dirichlet energy on the noise intensity.

图神经网络过度平滑随机扰动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。