针对异构数据下的个性化联邦学习,动态评估客户端贡献以提升模型性能。
CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks
- 基于梯度与预测偏差的双重分析,动态评估客户端更新质量。
- 在四个数据集上均优于现有方法,提升个性化准确率与收敛稳定性。
- 适合需要高鲁棒性个性化模型的分布式场景,如医疗或边缘计算。
个性化联邦学习(PFL)旨在解决异构且数据稀疏的客户端协同训练个性化模型的挑战。传统联邦学习依赖单一共识模型,其聚合方式通常按数据量或启发式加权,假设所有客户端贡献均等,忽视了实际更新的质量与可靠性,导致个性化效果差和聚合偏差。为此,本文提出贡献导向的个性化联邦学习(CO-PFL),通过联合分析梯度方向差异与预测偏差,从梯度与数据子空间中提取信息,为每个客户端动态生成具有判别性的聚合权重,突出高质量更新。此外,CO-PFL融合参数级个性化机制与掩码感知动量优化,增强个性化适应性与优化稳定性。在四个基准数据集(CIFAR10、CIFAR10C、CINIC10、Mini-ImageNet)上的实验表明,CO-PFL在个性化精度、鲁棒性、可扩展性和收敛稳定性方面持续超越当前最优方法。
原文摘要 · Abstract (English)
Personalized federated learning (PFL) addresses a critical challenge of collaboratively training customized models for clients with heterogeneous and scarce local data. Conventional federated learning, which relies on a single consensus model, proves inadequate under such data heterogeneity. Its standard aggregation method of weighting client updates heuristically or by data volume, operates under an equal-contribution assumption, failing to account for the actual utility and reliability of each client's update. This often results in suboptimal personalization and aggregation bias. To overcome these limitations, we introduce Contribution-Oriented PFL (CO-PFL), a novel algorithm that dynamically estimates each client's contribution for global aggregation. CO-PFL performs a joint assessment by analyzing both gradient direction discrepancies and prediction deviations, leveraging information from gradient and data subspaces. This dual-subspace analysis provides a principled and discriminative aggregation weight for each client, emphasizing high-quality updates. Furthermore, to bolster personalization adaptability and optimization stability, CO-PFL cohesively integrates a parameter-wise personalization mechanism with mask-aware momentum optimization. Our approach effectively mitigates aggregation bias, strengthens global coordination, and enhances local performance by facilitating the construction of tailored submodels with stable updates. Extensive experiments on four benchmark datasets (CIFAR10, CIFAR10C, CINIC10, and Mini-ImageNet) confirm that CO-PFL consistently surpasses state-of-the-art methods in in personalization accuracy, robustness, scalability and convergence stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。