用安德森局域化视角解释图神经网络过平滑问题
Rethinking Over-Smoothing in Graph Neural Networks: A Perspective from Anderson Localization
- 将过平滑类比为无序系统中的安德森局域化现象
- 提出参与度衡量节点特征同质化程度，随深度上升
- 为缓解过平滑提供新思路：降低信息传播的无序性
图神经网络（GNN）在图数据分析中展现出强大表征能力，但随着网络深度增加，过平滑问题愈发严重，导致节点表示失去区分性。本文通过类比安德森局域化，分析过平滑机制，并引入参与度作为量化指标。随着GNN层数增加，节点特征在多层消息传递后趋于同质化，类似无序系统中振动模式的行为。此时，过平滑表现为低频模式扩展（参与度上升）与高频模式局域化（参与度下降）。基于此，我们系统探讨了无序系统中的安德森局域化行为与GNN过平滑之间的潜在联系，并进行了理论分析，提出通过降低信息传播的无序性来缓解过平滑的潜力。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have shown great potential in graph data analysis due to their powerful representation capabilities. However, as the network depth increases, the issue of over-smoothing becomes more severe, causing node representations to lose their distinctiveness. This paper analyzes the mechanism of over-smoothing through the analogy to Anderson localization and introduces participation degree as a metric to quantify this phenomenon. Specifically, as the depth of the GNN increases, node features homogenize after multiple layers of message passing, leading to a loss of distinctiveness, similar to the behavior of vibration modes in disordered systems. In this context, over-smoothing in GNNs can be understood as the expansion of low-frequency modes (increased participation degree) and the localization of high-frequency modes (decreased participation degree). Based on this, we systematically reviewed the potential connection between the Anderson localization behavior in disordered systems and the over-smoothing behavior in Graph Neural Networks. A theoretical analysis was conducted, and we proposed the potential of alleviating over-smoothing by reducing the disorder in information propagation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。