arXiv:2411.01825cs.LGcs.DC2024-11中稿 · European Conferenc…被引 7

针对联邦学习中类别不平衡问题,提出动态选择相关客户端的个性化方法。

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

  • 通过分析客户端分类器输出相似性,动态识别任务相关客户端。
  • 分阶段使用不同聚合策略,提升个性化模型在不平衡数据上的表现。
  • 适合处理数据异构性强、需个性化建模的场景。

联邦学习(FL)是一种分布式机器学习范式,通过去中心化计算和周期性模型聚合,实现全局鲁棒模型,主要关注所有参与客户端聚合数据集上的全局模型精度。个性化联邦学习(PFL)则为每个客户端定制专属模型,旨在提升其在特定本地数据分布上的准确性。尽管广泛应用,现有FL与PFL研究尚未全面解决类别不平衡这一数据异构性中的关键挑战。本文提出FedReMa,一种高效的PFL算法,可通过两点缓解类别不平衡:1)采用自适应客户端间协同学习机制,在训练不同阶段识别并利用各客户端在不同数据类别上的专长;2)对特征提取器与分类器分别采用不同的聚合方式,依据其在模型中的角色与影响进行决策。基于对客户端间相似性动态的实验发现,我们设计了关键协同学习期(CCP),引入最大差异分割(MDS)模块,通过分析客户端分类器输出逻辑值间的相似性,评估并管理任务相关性。在CCP之外,进一步采用基于历史记录的最相关同伴聚合方案,以增强个性化稳定性。大量实验证明了FedReMa的优越性。

原文摘要 · Abstract (English)

Federated Learning (FL) is a distributed machine learning paradigm that achieves a globally robust model through decentralized computation and periodic model synthesis, primarily focusing on the global model's accuracy over aggregated datasets of all participating clients. Personalized Federated Learning (PFL) instead tailors exclusive models for each client, aiming to enhance the accuracy of clients' individual models on specific local data distributions. Despite of their wide adoption, existing FL and PFL works have yet to comprehensively address the class-imbalance issue, one of the most critical challenges within the realm of data heterogeneity in PFL and FL research. In this paper, we propose FedReMa, an efficient PFL algorithm that can tackle class-imbalance by 1) utilizing an adaptive inter-client co-learning approach to identify and harness different clients' expertise on different data classes throughout various phases of the training process, and 2) employing distinct aggregation methods for clients' feature extractors and classifiers, with the choices informed by the different roles and implications of these model components. Specifically, driven by our experimental findings on inter-client similarity dynamics, we develop critical co-learning period (CCP), wherein we introduce a module named maximum difference segmentation (MDS) to assess and manage task relevance by analyzing the similarities between clients' logits of their classifiers. Outside the CCP, we employ an additional scheme for model aggregation that utilizes historical records of each client's most relevant peers to further enhance the personalization stability. We demonstrate the superiority of our FedReMa in extensive experiments.

联邦学习个性化类别不平衡

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。