通过个性化子网络暖身提升极端数据异构下的联邦学习效果
FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning
- 每个参与方在初期仅训练特定子网络，避免冲突更新
- 实验显示准确率与收敛速度均优于传统方法
- 适合数据差异大的场景，如医疗、跨设备应用
统计上的数据异构是联邦学习（FL）收敛的主要障碍。尽管先前工作通过改进优化目标推进了异构联邦学习，但在参与者间存在极端数据异构时仍表现不佳。我们假设，在初始协作轮次中，由于参与者间冲突更新的聚合导致了收敛困难。为此，我们提出一个暖身阶段：每个参与者学习一个个性化掩码，仅更新模型的子网络。该个性化暖身使参与者能优先学习与其数据异构性匹配的特定子网络。暖身结束后，回归标准联邦优化，所有参数参与通信。实验证明，所提出的基于子网络的个性化暖身方法（FedPeWS）在准确率和收敛速度上均优于标准联邦优化方法。
原文摘要 · Abstract (English)
Statistical data heterogeneity is a significant barrier to convergence in federated learning (FL). While prior work has advanced heterogeneous FL through better optimization objectives, these methods fall short when there is extreme data heterogeneity among collaborating participants. We hypothesize that convergence under extreme data heterogeneity is primarily hindered due to the aggregation of conflicting updates from the participants in the initial collaboration rounds. To overcome this problem, we propose a warmup phase where each participant learns a personalized mask and updates only a subnetwork of the full model. This personalized warmup allows the participants to focus initially on learning specific subnetworks tailored to the heterogeneity of their data. After the warmup phase, the participants revert to standard federated optimization, where all parameters are communicated. We empirically demonstrate that the proposed personalized warmup via subnetworks (FedPeWS) approach improves accuracy and convergence speed over standard federated optimization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。