arXiv:2504.06768cs.LG2025-04AAAI被引 4

用模型合并实现个性化联邦学习,无需本地微调。

FedMerge: Federated Personalization via Model Merging

  • 通过自动优化权重合并多个全局模型,为每个客户端生成专属模型。
  • 在三个非独立同分布场景中表现优于聚类与专家混合方法。
  • 减少客户端漂移,平滑训练过程,适合多样任务的个性化需求。

联邦学习中的单一全局模型难以满足非独立同分布任务与数据分布的客户端需求。尽管已有研究尝试训练多个全局模型以提升个性化效果,但选择仍有限,本地微调仍不可或缺。本文提出一种新方法 FedMerge,通过自动优化并定制权重,将多个全局模型简单合并,为每个客户端生成个性化模型。该方法仅需服务器向各客户端发送定制化合并模型,无需周期性中断本地训练或重新初始化。合并模型更贴合客户端任务与数据分布,有效缓解因客户端漂移导致的局部-全局差距。我们在三种不同非独立同分布设置下对多种领域、任务和数据类型进行了评估,结果表明 FedMerge 在所有测试场景中均显著优于现有联邦学习方法,包括基于聚类和专家混合(MoE)的方法。

原文摘要 · Abstract (English)

One global model in federated learning (FL) might not be sufficient to serve many clients with non-IID tasks and distributions. While there has been advances in FL to train multiple global models for better personalization, they only provide limited choices to clients so local finetuning is still indispensable. In this paper, we propose a novel ``FedMerge'' approach that can create a personalized model per client by simply merging multiple global models with automatically optimized and customized weights. In FedMerge, a few global models can serve many non-IID clients, even without further local finetuning. We formulate this problem as a joint optimization of global models and the merging weights for each client. Unlike existing FL approaches where the server broadcasts one or multiple global models to all clients, the server only needs to send a customized, merged model to each client. Moreover, instead of periodically interrupting the local training and re-initializing it to a global model, the merged model aligns better with each client's task and data distribution, smoothening the local-global gap between consecutive rounds caused by client drift. We evaluate FedMerge on three different non-IID settings applied to different domains with diverse tasks and data types, in which FedMerge consistently outperforms existing FL approaches, including clustering-based and mixture-of-experts (MoE) based methods.

联邦学习模型合并个性化非IID

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。