只传梯度，不传数据，让图神经网络训练快4倍
Grappa: Gradient-Only Communication for Scalable Graph Neural Network Training
- 每轮只交换梯度，各分区独立训练，减少通信开销
- 在真实和合成图上平均提速4倍（最高13倍），支持万亿边规模
- 无需高速网络或缓存，适合深度模型和大规模部署
跨分区边是分布式图神经网络训练的主要瓶颈：每轮迭代中远程特征和激活值的获取消耗大量网络带宽，随着图加深和分区增多而愈发严重。Grappa 是一种分布式 GNN 训练框架，强制采用仅梯度通信：每轮迭代中各分区独立训练，仅交换用于全局更新的梯度。为弥补隔离导致的精度损失，Grappa（i）定期重新分区以暴露新邻域，（ii）采用轻量级覆盖修正梯度聚合，受重要性采样启发。我们提出了一个渐近无偏的梯度修正估计器，并据此开发了兼容主流深度学习框架的最小距离批次级变体。此外，还引入收缩版本以提升实际稳定性。在真实与合成图上的实证结果表明，Grappa 平均比现有最优系统快4倍（最高达13倍），尤其在深层模型下精度更高，且可在商品硬件上实现万亿边规模的训练。Grappa 与模型无关，支持全图与小批量训练，不依赖高带宽互连或缓存。
原文摘要 · Abstract (English)
Cross-partition edges dominate the cost of distributed GNN training: fetching remote features and activations per iteration overwhelms the network as graphs deepen and partition counts grow. Grappa is a distributed GNN training framework that enforces gradient-only communication: during each iteration, partitions train in isolation and exchange only gradients for the global update. To recover accuracy lost to isolation, Grappa (i) periodically repartitions to expose new neighborhoods and (ii) applies a lightweight coverage-corrected gradient aggregation inspired by importance sampling. We present an asymptotically unbiased estimator for gradient correction, which we use to develop a minimum-distance batch-level variant that is compatible with common deep-learning packages. We also introduce a shrinkage version that improves stability in practice. Empirical results on real and synthetic graphs show that Grappa trains GNNs 4x faster on average (up to 13x) than state-of-the-art systems, achieves better accuracy especially for deeper models, and sustains training at the trillion-edge scale on commodity hardware. Grappa is model-agnostic, supports full-graph and mini-batch training, and does not rely on high-bandwidth interconnects or caching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。