RapidGNN通过确定性采样提升大图神经网络分布式训练效率
RapidGNN: Energy and Communication-Efficient Distributed Training on Large-Scale Graph Neural Networks
- 采用确定性采样调度，优化远程特征缓存与预取
- 训练吞吐提升2.46至3.00倍，远程特征请求减少9.70至15.39倍
- 适合大规模图学习场景，显著降低通信开销和能耗
图神经网络（GNNs）在探索实体间结构关系的任务中广泛应用。然而，由于数据集高度连通，大规模图上的分布式训练面临巨大挑战。传统基于采样的方法虽缓解计算负担，但通信开销仍难克服。本文提出RapidGNN，一种基于确定性采样调度的分布式GNN训练框架，实现高效缓存构建与远程特征预取。在基准图数据集上的评估表明，RapidGNN在不同规模和拓扑下均有效：相比基线方法，端到端训练吞吐平均提升2.46倍至3.00倍，远程特征获取次数减少9.70倍至15.39倍；同时展现出近线性扩展性。此外，在CPU和GPU上分别实现44%和32%的能效提升。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have become popular across a diverse set of tasks in exploring structural relationships between entities. However, due to the highly connected structure of the datasets, distributed training of GNNs on large-scale graphs poses significant challenges. Traditional sampling-based approaches mitigate the computational loads, yet the communication overhead remains a challenge. This paper presents RapidGNN, a distributed GNN training framework with deterministic sampling-based scheduling to enable efficient cache construction and prefetching of remote features. Evaluation on benchmark graph datasets demonstrates RapidGNN's effectiveness across different scales and topologies. RapidGNN improves end-to-end training throughput by 2.46x to 3.00x on average over baseline methods across the benchmark datasets, while cutting remote feature fetches by over 9.70x to 15.39x. RapidGNN further demonstrates near-linear scalability with an increasing number of computing units efficiently. Furthermore, it achieves increased energy efficiency over the baseline methods for both CPU and GPU by 44% and 32%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。