提出渐进式参数对齐,提升异构设备下的个性化联邦学习效果
FedPPA: Progressive Parameter Alignment for Personalized Federated Learning
- 通过逐步对齐共用层权重,缓解全局与本地模型不一致
- 在MNIST、FMNIST、CIFAR-10上均优于现有方法,提升个性化适应能力
- 适合资源异构、数据非独立同分布的现实联邦学习场景
联邦学习(FL)是一种去中心化、保护隐私的机器学习范式,允许多个客户端在不共享数据的前提下协同训练模型。然而,在真实场景中,客户端往往具有异构的计算资源和非独立同分布(non-IID)的数据,给训练带来挑战。个性化联邦学习(PFL)通过为每个客户端定制模型来应对这一问题。现有方法通常忽视了因计算能力差异导致的模型与数据双重异构性。为此,我们提出一种新方法——渐进式参数对齐(FedPPA),通过逐步对齐各客户端共用层权重与全局模型权重,不仅缓解了客户端更新时全局与本地模型间的不一致,还保留了客户端的本地知识,增强了非IID环境下的个性化鲁棒性。为进一步提升全局模型性能并保持强个性化,我们在FedPPA框架中引入基于熵的加权平均策略。在三个图像分类数据集(MNIST、FMNIST、CIFAR-10)上的实验表明,FedPPA持续优于现有联邦学习算法,实现了更优的个性化适应性能。
原文摘要 · Abstract (English)
Federated Learning (FL) is designed as a decentralized, privacy-preserving machine learning paradigm that enables multiple clients to collaboratively train a model without sharing their data. In real-world scenarios, however, clients often have heterogeneous computational resources and hold non-independent and identically distributed data (non-IID), which poses significant challenges during training. Personalized Federated Learning (PFL) has emerged to address these issues by customizing models for each client based on their unique data distribution. Despite its potential, existing PFL approaches typically overlook the coexistence of model and data heterogeneity arising from clients with diverse computational capabilities. To overcome this limitation, we propose a novel method, called Progressive Parameter Alignment (FedPPA), which progressively aligns the weights of common layers across clients with the global model's weights. Our approach not only mitigates inconsistencies between global and local models during client updates, but also preserves client's local knowledge, thereby enhancing personalization robustness in non-IID settings. To further enhance the global model performance while retaining strong personalization, we also integrate entropy-based weighted averaging into the FedPPA framework. Experiments on three image classification datasets, including MNIST, FMNIST, and CIFAR-10, demonstrate that FedPPA consistently outperforms existing FL algorithms, achieving superior performance in personalized adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。