解决联邦学习中的数据异构问题，提升个性化模型性能。
A Novel Algorithm for Personalized Federated Learning: Knowledge Distillation with Weighted Combination Loss
- 用全局模型做教师，指导本地模型优化。
- 在MNIST和合成数据集上准确率更高，收敛更快。
- 适合数据分布差异大的场景，如医疗或金融领域。
联邦学习（FL）为分布式机器学习提供了一种保护隐私的框架，使不同客户端可在不集中敏感数据的情况下协同训练模型。然而，由非独立同分布（non-IID）客户端数据引起的统计异构性带来了显著挑战，导致模型漂移和泛化能力差。本文提出一种新算法pFedKD-WCL（带加权组合损失的个性化联邦知识蒸馏），将知识蒸馏与双层优化结合，以应对non-IID问题。该方法利用当前全局模型作为教师，指导本地模型训练，在保证全局收敛的同时实现高效本地个性化。我们在采用多项式逻辑回归和多层感知机的MNIST数据集及一个non-IID划分的合成数据集上进行了评估。实验结果表明，pFedKD-WCL在准确率和收敛速度方面均优于现有先进算法，包括FedAvg、FedProx、Per-FedAvg和pFedMe。
原文摘要 · Abstract (English)
Federated learning (FL) offers a privacy-preserving framework for distributed machine learning, enabling collaborative model training across diverse clients without centralizing sensitive data. However, statistical heterogeneity, characterized by non-independent and identically distributed (non-IID) client data, poses significant challenges, leading to model drift and poor generalization. This paper proposes a novel algorithm, pFedKD-WCL (Personalized Federated Knowledge Distillation with Weighted Combination Loss), which integrates knowledge distillation with bi-level optimization to address non-IID challenges. pFedKD-WCL leverages the current global model as a teacher to guide local models, optimizing both global convergence and local personalization efficiently. We evaluate pFedKD-WCL on the MNIST dataset and a synthetic dataset with non-IID partitioning, using multinomial logistic regression and multilayer perceptron models. Experimental results demonstrate that pFedKD-WCL outperforms state-of-the-art algorithms, including FedAvg, FedProx, Per-FedAvg, and pFedMe, in terms of accuracy and convergence speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。