解决联邦图学习中模型、数据、通信三重异构难题,提升性能并大幅降低通信开销。
Federated Prototype Graph Learning
- 客户端用多层级拓扑感知原型捕捉局部图语义
- 服务器通过对比学习个性化全局原型,提升训练效果
- 相比顶尖方法准确率平均提升3.57%,通信成本降低168倍
近年来,联邦图学习(FGL)因其在图智能应用中的分布式训练能力受到广泛关注,可在缓解数据孤岛的同时为隐私保护的大规模图学习提供新视角。然而,多层级的FGL异构性带来了客户端与服务器协作的挑战:(1) 模型层面:客户端对性能和可扩展性的需求差异要求部署异构模型,但多数FGL方法因直接在服务器端聚合模型权重而强制要求相同客户端模型;(2) 数据层面:图结构中节点属性与拓扑关系高度耦合,导致优化困难,使联邦训练模型难以获得优异性能;(3) 通信层面:部分方法试图增加客户端间或客户端与服务器之间的信息交换以提升训练效果,却不可避免带来高昂通信开销。本文提出FedPG,一种通用的原型引导优化方法,以应对上述多层级异构问题。具体而言,在客户端侧,引入多层级拓扑感知原型以捕捉局部图语义;在服务器侧,利用上传的原型,通过拓扑引导的对比学习与个性化技术为每个客户端定制全局原型,并广播回客户端以改进本地训练。实验表明,FedPG在准确率上平均优于主流基线3.57%,同时通信成本降低168倍。
原文摘要 · Abstract (English)
In recent years, Federated Graph Learning (FGL) has gained significant attention for its distributed training capabilities in graph-based machine intelligence applications, mitigating data silos while offering a new perspective for privacy-preserve large-scale graph learning. However, multi-level FGL heterogeneity presents various client-server collaboration challenges: (1) Model-level: The variation in clients for expected performance and scalability necessitates the deployment of heterogeneous models. Unfortunately, most FGL methods rigidly demand identical client models due to the direct model weight aggregation on the server. (2) Data-level: The intricate nature of graphs, marked by the entanglement of node profiles and topology, poses an optimization dilemma. This implies that models obtained by federated training struggle to achieve superior performance. (3) Communication-level: Some FGL methods attempt to increase message sharing among clients or between clients and the server to improve training, which inevitably leads to high communication costs. In this paper, we propose FedPG as a general prototype-guided optimization method for the above multi-level FGL heterogeneity. Specifically, on the client side, we integrate multi-level topology-aware prototypes to capture local graph semantics. Subsequently, on the server side, leveraging the uploaded prototypes, we employ topology-guided contrastive learning and personalized technology to tailor global prototypes for each client, broadcasting them to improve local training. Experiments demonstrate that FedPG outperforms SOTA baselines by an average of 3.57\% in accuracy while reducing communication costs by 168x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。