提出新方法可视化GNN损失曲面，揭示训练优化关键影响因素。
Visualization and Analysis of the Loss Landscape in Graph Neural Networks
- 用可学习投影替代PCA，高效还原高维参数空间
- 发现架构、稀疏化和预条件对优化曲面影响显著
- 适合研究GNN训练机制或设计更优模型的读者
图神经网络（GNN）在图结构数据上表现强大，但其参数优化、表达能力与泛化性能之间的关系仍不清晰。本文提出一种高效的可学习降维方法，用于可视化GNN的损失景观，并分析过平滑、跳跃知识、量化、稀疏化及预条件器对GNN优化的影响。所提可学习投影方法优于现有基于PCA的方法，可在更低内存下准确重建高维参数空间。研究进一步表明，网络架构、稀疏化策略以及优化器的预条件机制显著影响GNN的优化景观、训练过程及最终预测性能。这些发现有助于设计更高效的GNN架构与训练策略。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) are powerful models for graph-structured data, with broad applications. However, the interplay between GNN parameter optimization, expressivity, and generalization remains poorly understood. We address this by introducing an efficient learnable dimensionality reduction method for visualizing GNN loss landscapes, and by analyzing the effects of over-smoothing, jumping knowledge, quantization, sparsification, and preconditioner on GNN optimization. Our learnable projection method surpasses the state-of-the-art PCA-based approach, enabling accurate reconstruction of high-dimensional parameters with lower memory usage. We further show that architecture, sparsification, and optimizer's preconditioning significantly impact the GNN optimization landscape and their training process and final prediction performance. These insights contribute to developing more efficient designs of GNN architectures and training strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。