动态建模客户端协作关系,提升联邦学习个性化效果
FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks
- 用注意力图超网络动态构建客户端协作图
- 在多个数据集上实现比基线更高的个性化性能
- 适合研究个性化联邦学习与图神经网络的学者
个性化联邦学习(PFL)旨在通过为每个客户端学习专属模型来应对数据的统计异质性。现有基于个性化的聚合方法在服务器端进行参数聚合以生成个性化模型,重点在于学习客户端间的协同关系。然而,协同关系会随场景和训练阶段变化。为此,我们提出基于注意力图超网络的个性化联邦学习方法(FedAGHN),利用注意力图超网络(AGHN)动态捕捉细粒度的协同关系,并生成客户端特定的个性化初始模型。具体而言,AGHN 显式建模客户端间的协同关系,构建协作图,并引入可调注意力机制计算协同权重,从而通过协作图上的参数聚合获得个性化初始模型。大量实验表明 FedAGHN 的优越性;此外,一系列可视化分析展示了其学习到的协作图的有效性。
原文摘要 · Abstract (English)
Personalized Federated Learning (PFL) aims to address the statistical heterogeneity of data across clients by learning the personalized model for each client. Among various PFL approaches, the personalized aggregation-based approach conducts parameter aggregation in the server-side aggregation phase to generate personalized models, and focuses on learning appropriate collaborative relationships among clients for aggregation. However, the collaborative relationships vary in different scenarios and even at different stages of the FL process. To this end, we propose Personalized Federated Learning with Attentive Graph HyperNetworks (FedAGHN), which employs Attentive Graph HyperNetworks (AGHNs) to dynamically capture fine-grained collaborative relationships and generate client-specific personalized initial models. Specifically, AGHNs empower graphs to explicitly model the client-specific collaborative relationships, construct collaboration graphs, and introduce tunable attentive mechanism to derive the collaboration weights, so that the personalized initial models can be obtained by aggregating parameters over the collaboration graphs. Extensive experiments can demonstrate the superiority of FedAGHN. Moreover, a series of visualizations are presented to explore the effectiveness of collaboration graphs learned by FedAGHN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。