通过聚合头部提升个性化联邦学习性能,兼顾本地特性和全局知识。
FedAH: Aggregated Head for Personalized Federated Learning
- 每轮迭代用全局头部聚合本地头部,融合全局信息
- 在5个基准数据集上测试准确率领先其他方法2.87%
- 适合客户端不稳定场景,抗掉线能力强
联邦学习(FL)因其隐私保护和协作学习能力受到关注。个性化联邦学习(PFL)旨在解决统计异质性问题,其中基于个性化头部的方法将模型分为共享的特征提取器和本地训练的头部。但仅保留本地头部会阻碍头部学习全局知识,影响性能。为此,我们提出一种新方法FedAH,每轮迭代用全局头部聚合本地头部,实现元素级聚合,引入全局信息。在计算机视觉与自然语言处理领域的五个基准数据集上进行实验,FedAH在测试准确率上比十种先进方法平均高出2.87%。即使部分客户端意外掉线,仍保持优势。代码已开源:https://github.com/heyuepeng/FedAH。
原文摘要 · Abstract (English)
Recently, Federated Learning (FL) has gained popularity for its privacy-preserving and collaborative learning capabilities. Personalized Federated Learning (PFL), building upon FL, aims to address the issue of statistical heterogeneity and achieve personalization. Personalized-head-based PFL is a common and effective PFL method that splits the model into a feature extractor and a head, where the feature extractor is collaboratively trained and shared, while the head is locally trained and not shared. However, retaining the head locally, although achieving personalization, prevents the model from learning global knowledge in the head, thus affecting the performance of the personalized model. To solve this problem, we propose a novel PFL method called Federated Learning with Aggregated Head (FedAH), which initializes the head with an Aggregated Head at each iteration. The key feature of FedAH is to perform element-level aggregation between the local model head and the global model head to introduce global information from the global model head. To evaluate the effectiveness of FedAH, we conduct extensive experiments on five benchmark datasets in the fields of computer vision and natural language processing. FedAH outperforms ten state-of-the-art FL methods in terms of test accuracy by 2.87%. Additionally, FedAH maintains its advantage even in scenarios where some clients drop out unexpectedly. Our code is open-accessed at https://github.com/heyuepeng/FedAH.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。