FedQS统一优化异步联邦学习的梯度与模型聚合,提升精度与收敛速度。
FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning
- 按客户端类型分组,动态调整训练策略以应对数据异构性。
- 在多个任务上实现最高准确率、最低损失和最快收敛速度。
- 适合需要稳定高效训练的工业级联邦学习场景。
联邦学习(FL)允许多方协作训练模型而不共享原始数据,半异步联邦学习(SAFL)成为同步与异步之间的平衡方案。然而,SAFL在优化基于梯度(如FedSGD)和基于模型(如FedAvg)的聚合策略时面临显著挑战,二者在精度、收敛速度和稳定性上各有优劣:梯度聚合收敛快、精度高但波动大;模型聚合更稳定但收敛慢且精度不足。本文提出首个理论分析并解决此差异的框架FedQS。FedQS采用分而治之策略,将客户端分为四类,依据数据分布特征和计算资源自适应优化本地训练。在计算机视觉、自然语言处理及真实任务上的大量实验表明,FedQS在所有基准中达到最高精度、最低损失,并位列最快收敛速度,优于现有最优方法。本工作弥合了SAFL中聚合策略的差距,提供统一的稳定、精准、高效的联邦学习解决方案。代码与数据集见https://github.com/bkjod/FedQS_。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative model training across multiple parties without sharing raw data, with semi-asynchronous FL (SAFL) emerging as a balanced approach between synchronous and asynchronous FL. However, SAFL faces significant challenges in optimizing both gradient-based (e.g., FedSGD) and model-based (e.g., FedAvg) aggregation strategies, which exhibit distinct trade-offs in accuracy, convergence speed, and stability. While gradient aggregation achieves faster convergence and higher accuracy, it suffers from pronounced fluctuations, whereas model aggregation offers greater stability but slower convergence and suboptimal accuracy. This paper presents FedQS, the first framework to theoretically analyze and address these disparities in SAFL. FedQS introduces a divide-and-conquer strategy to handle client heterogeneity by classifying clients into four distinct types and adaptively optimizing their local training based on data distribution characteristics and available computational resources. Extensive experiments on computer vision, natural language processing, and real-world tasks demonstrate that FedQS achieves the highest accuracy, attains the lowest loss, and ranks among the fastest in convergence speed, outperforming state-of-the-art baselines. Our work bridges the gap between aggregation strategies in SAFL, offering a unified solution for stable, accurate, and efficient federated learning. The code and datasets are available at https://github.com/bkjod/FedQS_.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。