arXiv:2503.05755cs.DCcs.AI2025-03被引 3

提出SEAFL框架,解决联邦学习中慢节点和过时模型问题。

SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Training

  • 根据模型更新的过时程度与重要性动态加权聚合
  • 在三个数据集上实现比最接近对比方法快约22%的训练速度
  • 适合存在异构设备、通信不稳定的实际联邦学习场景

联邦学习(FL)是一种分布式机器学习框架,可在不上传本地数据的情况下让分散设备协同训练全局模型。然而,在真实场景中,传统同步联邦学习因慢速设备(即‘拖后腿者’)导致训练效率低下,尤其在异构通信环境下更为明显。尽管异步联邦学习能缓解效率问题,但会引入高系统开销和模型性能下降。为兼顾二者,半异步联邦学习受到关注,但仍面临过时模型的挑战:新上传的更新基于陈旧权重,易损害全局模型收敛。本文提出SEAFL框架,通过动态调整上传模型在聚合时的权重来缓解拖后腿者和过时模型问题,该权重依据模型的过时程度和对当前全局模型的重要性确定。我们理论分析了SEAFL的收敛速率,并设计扩展变体,允许在慢速设备上进行部分训练,使其仍可参与全局聚合,同时减少等待时间。在三个基准数据集上的大量实验表明,SEAFL在达到目标准确率所需的实际训练时间上,相比最接近的对比方法提升约22%。

原文摘要 · Abstract (English)

Federated Learning (FL) is a promising distributed machine learning framework that allows collaborative learning of a global model across decentralized devices without uploading their local data. However, in real-world FL scenarios, the conventional synchronous FL mechanism suffers from inefficient training caused by slow-speed devices, commonly known as stragglers, especially in heterogeneous communication environments. Though asynchronous FL effectively tackles the efficiency challenge, it induces substantial system overheads and model degradation. Striking for a balance, semi-asynchronous FL has gained increasing attention, while still suffering from the open challenge of stale models, where newly arrived updates are calculated based on outdated weights that easily hurt the convergence of the global model. In this paper, we present {\em SEAFL}, a novel FL framework designed to mitigate both the straggler and the stale model challenges in semi-asynchronous FL. {\em SEAFL} dynamically assigns weights to uploaded models during aggregation based on their staleness and importance to the current global model. We theoretically analyze the convergence rate of {\em SEAFL} and further enhance the training efficiency with an extended variant that allows partial training on slower devices, enabling them to contribute to global aggregation while reducing excessive waiting times. We evaluate the effectiveness of {\em SEAFL} through extensive experiments on three benchmark datasets. The experimental results demonstrate that {\em SEAFL} outperforms its closest counterpart by up to $\sim$22\% in terms of the wall-clock training time required to achieve target accuracy.

联邦学习半异步模型聚合效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。