arXiv:2502.12608cs.LGcs.AI2025-02KDD被引 5

首次揭示图神经网络的模式连通性,发现图结构决定优化特性。

Unveiling Mode Connectivity in Graph Neural Networks

  • 通过实证分析图神经网络的损失曲面几何,发现其非线性模式连通性。
  • 图同质性等属性显著影响模式连通性,优于模型架构本身。
  • 该现象可解释领域对齐策略,并为训练优化提供理论依据。

理解图神经网络(GNN)的优化动态与损失曲面几何是提升可解释性与鲁棒性的关键挑战。尽管模式连通性在其他深度学习架构中已被证明对损失曲面几何具有洞察力,但其在GNN中的作用尚未被探索。本文首次系统研究了GNN中的模式连通性,发现其呈现独特的非线性特征,不同于全连接网络或卷积网络。关键发现是:图结构而非模型架构主导该行为,图同质性等属性与模式连通性模式密切相关。我们进一步建立模式连通性与泛化能力的联系,提出基于损失屏障的泛化界,并验证其作为诊断工具的有效性。研究结果将理论洞见与实践应用相衔接,为图学习中的领域对齐策略提供解释,并奠定改进GNN训练范式的基础。

原文摘要 · Abstract (English)

A fundamental challenge in understanding graph neural networks (GNNs) lies in characterizing their optimization dynamics and loss landscape geometry, critical for improving interpretability and robustness. While mode connectivity, a lens for analyzing geometric properties of loss landscapes has proven insightful for other deep learning architectures, its implications for GNNs remain unexplored. This work presents the first investigation of mode connectivity in GNNs. We uncover that GNNs exhibit distinct non-linear mode connectivity, diverging from patterns observed in fully-connected networks or CNNs. Crucially, we demonstrate that graph structure, rather than model architecture, dominates this behavior, with graph properties like homophily correlating with mode connectivity patterns. We further establish a link between mode connectivity and generalization, proposing a generalization bound based on loss barriers and revealing its utility as a diagnostic tool. Our findings further bridge theoretical insights with practical implications: they rationalize domain alignment strategies in graph learning and provide a foundation for refining GNN training paradigms.

图神经网络优化几何模式连通性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。