用动力系统理论解决图神经网络过平滑问题
A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networks
- 从动力系统角度分析过平滑成因，设计动态剪枝策略
- 在多个基准数据集上提升深层GNN性能与稳定性
- 适合研究图神经网络深度优化的学者和工程师
图神经网络（GNN）随着网络深度增加面临过平滑问题，导致节点表示趋同、表达能力下降。本文从动力系统视角出发，深入理解GNN的稳定性和收敛行为，揭示过平滑的根本原因。基于此，提出DYNAMO-GAT方法，利用噪声驱动的协方差分析与反赫布原则，动态剪枝冗余注意力权重，有效维持节点特征多样性与网络稳定性。理论分析表明，DYNAMO-GAT能阻断向过平滑状态的收敛。实验结果在多个基准数据集上验证了其优于传统及先进方法的性能与效率。该工作不仅深化了对过平滑现象的动力系统理解，也提供了实用高效的深层GNN稳定化解决方案。
原文摘要 · Abstract (English)
Oversmoothing in Graph Neural Networks (GNNs) poses a significant challenge as network depth increases, leading to homogenized node representations and a loss of expressiveness. In this work, we approach the oversmoothing problem from a dynamical systems perspective, providing a deeper understanding of the stability and convergence behavior of GNNs. Leveraging insights from dynamical systems theory, we identify the root causes of oversmoothing and propose \textbf{\textit{DYNAMO-GAT}}. This approach utilizes noise-driven covariance analysis and Anti-Hebbian principles to selectively prune redundant attention weights, dynamically adjusting the network's behavior to maintain node feature diversity and stability. Our theoretical analysis reveals how DYNAMO-GAT disrupts the convergence to oversmoothed states, while experimental results on benchmark datasets demonstrate its superior performance and efficiency compared to traditional and state-of-the-art methods. DYNAMO-GAT not only advances the theoretical understanding of oversmoothing through the lens of dynamical systems but also provides a practical and effective solution for improving the stability and expressiveness of deep GNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。