通过自适应层混合法提升个性化联邦学习的效率与鲁棒性
pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup
- 在全局与本地模型间动态进行分层混合作用,实现灵活个性化
- 相比现有方法,训练更快、抗数据异质性更强,准确率更高
- 适合需要快速适配异构数据的个性化场景,如医疗或移动端应用
传统联邦学习在处理非独立同分布(non-IID)数据时面临显著挑战,难以提供个性化解决方案。个性化联邦学习(PFL)通过参数解耦或部分冻结神经网络层来平衡泛化与个性化,但现有方法仍存在全局-局部模型差异、客户端漂移和灾难性遗忘等问题,导致性能下降。为此,我们提出pMixFed,一种动态分层的个性化联邦学习方法,将共享全局模型与个性化本地模型之间引入mixup机制。该方法采用自适应策略划分个性化与共享层,逐步调整个性化程度以增强客户端适应能力,提升跨客户端泛化性能,并设计新型聚合机制缓解灾难性遗忘。大量实验表明,pMixFed优于当前最优PFL方法,在不同异构设置下展现出更快训练速度、更强鲁棒性及更优的数据异质性处理能力。
原文摘要 · Abstract (English)
Traditional Federated Learning (FL) methods encounter significant challenges when dealing with heterogeneous data and providing personalized solutions for non-IID scenarios. Personalized Federated Learning (PFL) approaches aim to address these issues by balancing generalization and personalization, often through parameter decoupling or partial models that freeze some neural network layers for personalization while aggregating other layers globally. However, existing methods still face challenges of global-local model discrepancy, client drift, and catastrophic forgetting, which degrade model accuracy. To overcome these limitations, we propose $\textit{pMixFed}$, a dynamic, layer-wise PFL approach that integrates $\textit{mixup}$ between shared global and personalized local models. Our method introduces an adaptive strategy for partitioning between personalized and shared layers, a gradual transition of personalization degree to enhance local client adaptation, improved generalization across clients, and a novel aggregation mechanism to mitigate catastrophic forgetting. Extensive experiments demonstrate that pMixFed outperforms state-of-the-art PFL methods, showing faster model training, increased robustness, and improved handling of data heterogeneity under different heterogeneous settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。