通过远程邻居剪枝与计算通信重叠，加速联邦图神经网络训练
Optimizing Federated Learning using Remote Embeddings for Graph Neural Networks
- 用远程邻居剪枝减少上传数据量，结合计算与通信重叠降低开销
- 在Reddit和Products数据集上训练速度提升约2倍，准确率提高20%
- 适合大规模密集图数据的隐私保护分布式学习场景
图神经网络（GNN）近年来发展迅速，能从图结构数据中学习有意义的表示。联邦学习（FL）作为一种在分散数据上训练共享模型的可行方法，兼顾隐私保护与并行性。现有利用远程嵌入提升联邦GNN收敛精度的方法受限于共享嵌入服务器带来的高通信成本。本文提出OpES框架，采用远程邻居剪枝，并将嵌入推送与本地训练重叠，显著降低网络开销与训练时间。尽管每轮提前推送嵌入导致精度轻微下降，但在Reddit和Products等大规模密集图上，整体训练速度比现有最优技术快约2倍，且准确率较原始联邦GNN提升最高达20%。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have experienced rapid advancements in recent years due to their ability to learn meaningful representations from graph data structures. Federated Learning (FL) has emerged as a viable machine learning approach for training a shared model on decentralized data, addressing privacy concerns while leveraging parallelism. Existing methods that address the unique requirements of federated GNN training using remote embeddings to enhance convergence accuracy are limited by their diminished performance due to large communication costs with a shared embedding server. In this paper, we present OpES, an optimized federated GNN training framework that uses remote neighbourhood pruning, and overlaps pushing of embeddings to the server with local training to reduce the network costs and training time. The modest drop in per-round accuracy due to pre-emptive push of embeddings is out-stripped by the reduction in per-round training time for large and dense graphs like Reddit and Products, converging up to $\approx2\times$ faster than the state-of-the-art technique using an embedding server and giving up to $20\%$ better accuracy than vanilla federated GNN learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。