arXiv:2504.04081cs.LGcs.DC2025-04中稿 · as a full paper at…被引 4

用知识蒸馏修正异步联邦学习中的模型版本差异,提升训练效率与准确率。

Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible

  • 利用最新全局模型对过时梯度进行知识蒸馏修正。
  • 在不同训练阶段自适应调整蒸馏权重,避免早期误导。
  • 显著加快收敛速度,适合高延迟或不均衡设备的联邦学习场景。

异步联邦学习相比传统同步方法具有显著提速优势,但因客户端模型版本不一致,过时梯度可能导致聚合冲突,降低模型精度。本文提出基于知识蒸馏的异步联邦学习版本校正算法FedADT,通过在梯度聚合前使用最新全局模型修正过时信息,有效缓解版本差异带来的负面影响。同时引入自适应加权函数,根据训练阶段动态调整蒸馏强度,减轻早期全局模型性能较差导致的误导。实验表明,该方法在多个基准数据集上优于现有异步算法,显著提升收敛速度与最终性能,且计算开销可控。

原文摘要 · Abstract (English)

As an emerging paradigm of federated learning, asynchronous federated learning offers significant speed advantages over traditional synchronous federated learning. Unlike synchronous federated learning, which requires waiting for all clients to complete updates before aggregation, asynchronous federated learning aggregates the models that have arrived in realtime, greatly improving training speed. However, this mechanism also introduces the issue of client model version inconsistency. When the differences between models of different versions during aggregation become too large, it may lead to conflicts, thereby reducing the models accuracy. To address this issue, this paper proposes an asynchronous federated learning version correction algorithm based on knowledge distillation, named FedADT. FedADT applies knowledge distillation before aggregating gradients, using the latest global model to correct outdated information, thus effectively reducing the negative impact of outdated gradients on the training process. Additionally, FedADT introduces an adaptive weighting function that adjusts the knowledge distillation weight according to different stages of training, helps mitigate the misleading effects caused by the poorer performance of the global model in the early stages of training. This method significantly improves the overall performance of asynchronous federated learning without adding excessive computational overhead. We conducted experimental comparisons with several classical algorithms, and the results demonstrate that FedADT achieves significant improvements over other asynchronous methods and outperforms all methods in terms of convergence speed.

联邦学习异步训练知识蒸馏模型一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。