FedDPC通过投影与自适应缩放,缓解联邦学习中的数据异构与部分客户端参与问题。
FedDPC : Handling Data Heterogeneity and Partial Client Participation in Federated Learning
- 将本地更新投影到前一全局更新方向,抑制更新方差
- 在多个异构数据集上测试,训练损失下降更快,准确率更高
- 适合数据分布不均且客户端常缺席的现实联邦场景
数据异构性是现代联邦学习(FL)的重大挑战,导致本地模型更新存在方差,使聚合后的全局模型偏离真实全局最优。部分客户端参与进一步加剧此问题,使全局模型聚合偏向活跃客户端的数据分布,引入额外方差,导致全局模型向全局目标最优值收敛困难。这些方差引发训练不稳定,降低模型性能并减缓收敛速度。现有研究多关注数据异构性,对部分客户端参与的影响关注较少。本文提出 FedDPC,一种新型联邦学习方法,通过同时缓解数据异构性和部分客户端参与问题,提升训练效率和全局模型性能。其核心机制为:将每个本地更新投影至前一全局更新方向,从而控制本地与全局更新的方差;并通过自适应缩放各本地更新后进行聚合,加速训练。在多个异构划分图像分类数据集上的实验验证了 FedDPC 的有效性,结果表明其相比现有最优算法,在通信轮次中实现更快速的训练损失下降与更高的测试准确率。
原文摘要 · Abstract (English)
Data heterogeneity is a significant challenge in modern federated learning (FL) as it creates variance in local model updates, causing the aggregated global model to shift away from the true global optimum. Partial client participation in FL further exacerbates this issue by skewing the aggregation of local models towards the data distribution of participating clients. This creates additional variance in the global model updates, causing the global model to converge away from the optima of the global objective. These variances lead to instability in FL training, which degrades global model performance and slows down FL training. While existing literature primarily focuses on addressing data heterogeneity, the impact of partial client participation has received less attention. In this paper, we propose FedDPC, a novel FL method, designed to improve FL training and global model performance by mitigating both data heterogeneity and partial client participation. FedDPC addresses these issues by projecting each local update onto the previous global update, thereby controlling variance in both local and global updates. To further accelerate FL training, FedDPC employs adaptive scaling for each local update before aggregation. Extensive experiments on image classification tasks with multiple heterogeneously partitioned datasets validate the effectiveness of FedDPC. The results demonstrate that FedDPC outperforms state-of-the-art FL algorithms by achieving faster reduction in training loss and improved test accuracy across communication rounds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。