arXiv:2601.22678cs.LG2026-01被引 1

对比图神经网络全图与小批量训练,揭示批大小和扇出大小的影响。

Full-Graph vs. Mini-Batch Training: Comprehensive Analysis from a Batch Size and Fan-Out Size Perspective

  • 用Wasserstein距离分析图结构对泛化的影响,引入扇出大小新视角。
  • 发现批大小和扇出大小影响非对称,小批量调优可超越全图训练。
  • 适合资源受限场景下优化GNN训练策略的研究者参考。

全图与小批量图神经网络(GNN)训练方法在系统设计需求上差异显著,选择合适方法至关重要。比较两种训练方式的核心挑战在于刻画模型性能(收敛性与泛化能力)和计算效率。虽然批大小在深度神经网络(DNN)分析中已成有效视角,但GNN进一步引入了扇出大小,因全图训练可视为具有最大批大小和扇出大小的小批量训练。然而,批大小与扇出大小对GNN的影响仍不充分探索。本文从批大小与扇出大小的视角,通过实证与理论分析系统比较全图与小批量训练。关键贡献包括:1)提出基于Wasserstein距离的泛化分析,研究图结构尤其是扇出大小的影响;2)揭示批大小与扇出大小在GNN收敛与泛化中的非各向同性效应,为资源受限条件下的超参数调优提供实践指导。最终表明,全图训练并不总优于精心调优的小批量设置。代码见:https://github.com/LIUMENGFAN-gif/GNN_fullgraph_minibatch_training。

原文摘要 · Abstract (English)

Full-graph and mini-batch Graph Neural Network (GNN) training approaches have distinct system design demands, making it crucial to choose the appropriate approach to develop. A core challenge in comparing these two GNN training approaches lies in characterizing their model performance (i.e., convergence and generalization) and computational efficiency. While a batch size has been an effective lens in analyzing such behaviors in deep neural networks (DNNs), GNNs extend this lens by introducing a fan-out size, as full-graph training can be viewed as mini-batch training with the largest possible batch size and fan-out size. However, the impact of the batch and fan-out size for GNNs remains insufficiently explored. To this end, this paper systematically compares full-graph vs. mini-batch training of GNNs through empirical and theoretical analyses from the view points of the batch size and fan-out size. Our key contributions include: 1) We provide a novel generalization analysis using the Wasserstein distance to study the impact of the graph structure, especially the fan-out size. 2) We uncover the non-isotropic effects of the batch size and the fan-out size in GNN convergence and generalization, providing practical guidance for tuning these hyperparameters under resource constraints. Finally, full-graph training does not always yield better model performance or computational efficiency than well-tuned smaller mini-batch settings. The implementation can be found in the github link: https://github.com/LIUMENGFAN-gif/GNN_fullgraph_minibatch_training.

图神经网络训练策略批大小扇出大小

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。