arXiv:2412.04416cs.LGcs.AI2024-12被引 3

针对联邦学习中数据异构导致的模型性能下降问题,提出双策略自适应优化方案。

FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning

  • 客户端采用自适应损失函数,平衡本地优化与全局一致性。
  • 服务器动态聚合各客户端模型,适配不同学习模式。
  • 在三个真实数据集上验证有效,收敛更快且更鲁棒。

联邦学习通过整合多个客户端的本地模型实现分布式训练,虽保护了数据隐私,但面临因客户端数据分布异构导致的性能下降、收敛变慢及全局模型鲁棒性降低等问题。其中标签偏移是图像分类等场景下的主要挑战。本文通过全面实验定位训练过程中的根本问题,并提出一种双策略方法:首先设计自适应损失函数,使客户端在保留已有知识的同时维持本地优化与全局一致性的最佳平衡;其次开发动态聚合策略,根据各客户端独特学习模式调整聚合方式,有效缓解网络中数据多样性带来的挑战。在三个真实世界数据集上的综合评估,结合理论收敛性证明,表明该方法显著优于多种现有先进方法。

原文摘要 · Abstract (English)

Federated Learning (FL) marks a transformative approach to distributed model training by combining locally optimized models from various clients into a unified global model. While FL preserves data privacy by eliminating centralized storage, it encounters significant challenges such as performance degradation, slower convergence, and reduced robustness of the global model due to the heterogeneity in client data distributions. Among the various forms of data heterogeneity, label skew emerges as a particularly formidable and prevalent issue, especially in domains such as image classification. To address these challenges, we begin with comprehensive experiments to pinpoint the underlying issues in the FL training process. Based on our findings, we then introduce an innovative dual-strategy approach designed to effectively resolve these issues. First, we introduce an adaptive loss function for client-side training, meticulously crafted to preserve previously acquired knowledge while maintaining an optimal equilibrium between local optimization and global model coherence. Secondly, we develop a dynamic aggregation strategy for aggregating client models at the server. This approach adapts to each client's unique learning patterns, effectively addressing the challenges of diverse data across the network. Our comprehensive evaluation, conducted across three diverse real-world datasets, coupled with theoretical convergence guarantees, demonstrates the superior efficacy of our method compared to several established state-of-the-art approaches.

联邦学习数据异构自适应优化模型聚合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。