Plexus通过三维并行训练，实现百亿边图的高效全图GNN训练。
Plexus: Taming Billion-edge Graphs with 3D Parallel Full-graph GNN Training
- 提出3D并行框架，解决图神经网络训练中的通信开销与负载不均问题。
- 在Perlmutter和Frontier上实现2048和1024块GPU扩展，速度提升2.3-12.5倍。
- 适合大规模图数据、需全图训练且追求极致性能的研究者与工程师。
图神经网络（GNN）利用真实世界图的连接性和结构来学习节点间的复杂属性与关系。许多现实图因规模过大，超出单个GPU内存容量，传统训练依赖小批量采样进行扩展。而分布式全图训练则因图结构的不规则性，面临高通信开销和负载不均衡问题。本文提出一种三维（3D）并行全图训练方法，有效缓解上述挑战，并支持百亿边图的训练。我们引入双重排列策略以实现负载均衡，并构建性能模型预测最优3D配置——即Plexus系统。在六个不同图数据集上评估，Plexus在Perlmutter系统上实现最高2048张GPU扩展，在Frontier上达1024张GPU扩展。相比现有最先进方法，Plexus实现2.3-12.5倍加速，时间到解缩短5.2-8.7倍（Perlmutter）和7.0-54.2倍（Frontier）。
原文摘要 · Abstract (English)
Graph neural networks (GNNs) leverage the connectivity and structure of real-world graphs to learn intricate properties and relationships between nodes. Many real-world graphs exceed the memory capacity of a GPU due to their sheer size, and training GNNs on such graphs requires techniques such as mini-batch sampling to scale. The alternative approach of distributed full-graph training suffers from high communication overheads and load imbalance due to the irregular structure of graphs. We propose a three-dimensional (3D) parallel approach for full-graph training that tackles these issues and scales to billion-edge graphs. In addition, we introduce optimizations such as a double permutation scheme for load balancing, and a performance model to predict the optimal 3D configuration of our parallel implementation -- Plexus. We evaluate Plexus on six different graph datasets and show scaling results on up to 2048 GPUs of Perlmutter, and 1024 GPUs of Frontier. Plexus achieves unprecedented speedups of 2.3-12.5x over prior state of the art, and a reduction in time-to-solution by 5.2-8.7x on Perlmutter and 7.0-54.2x on Frontier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。