通过感知图结构的随机采样,显著提升GNN训练效率。
Efficient GNN Training Through Structure-Aware Randomized Mini-Batching
- 基于社区结构设计随机化采样策略,平衡效率与精度。
- 平均提速1.8倍,最高峰值达2.76倍,精度损失仅0.42%。
- 适合追求训练速度且对精度要求较高的GNN应用开发者。
图神经网络(GNN)在真实世界图数据上实现学习,小批量训练已成为主流方法,因其可扩展至超大规模图并加速收敛。然而现有小批量构建策略普遍忽视训练效率,随机化方案常忽略图的结构性质(如社区结构),导致内存访问不规则,难以充分利用GPU片上缓存。而纯基于图结构的确定性采样虽运行快,却因缺乏随机性影响模型准确率和收敛速度。本文提出社区结构感知的随机小批量方法(COMM-RAND),在完全随机与完全结构依赖之间灵活权衡,实现更高效的GNN训练。我们在四个主流图学习基准上评估,COMM-RAND 最高可减少2.76倍训练时间(平均1.8倍),同时精度仅比主流随机方法低0.42%(最高1.79%点)。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) enable learning on realworld graphs and mini-batch training has emerged as the de facto standard for training GNNs because it can scale to very large graphs and improve convergence. Current mini-batch construction policies largely ignore efficiency considerations of GNN training. Specifically, existing mini-batching techniques employ randomization schemes to improve accuracy and convergence. However, these randomization schemes are often agnostic to the structural properties of the graph (for eg. community structure), resulting in highly irregular memory access patterns during GNN training that make suboptimal use of on-chip GPU caches. On the other hand, while deterministic mini-batching based solely on graph structure delivers fast runtime performance, the lack of randomness compromises both the final model accuracy and training convergence speed. In this paper, we present Community-structure-aware Randomized Mini-batching (COMM-RAND), a novel methodology that bridges the gap between the above extremes. COMM-RAND allows practitioners to explore the space between pure randomness and pure graph structural awareness during mini-batch construction, leading to significantly more efficient GNN training with similar accuracy. We evaluated COMM-RAND across four popular graph learning benchmarks. COMM-RAND cuts down GNN training time by up to 2.76x (1.8x on average) while achieving an accuracy that is within 1.79% points (0.42% on average) compared to popular random mini-batching approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。