用更优距离度量提升异步联邦学习的收敛与稳定性
Revisiting Gradient Staleness: Evaluating Distance Metrics for Asynchronous Federated Learning Aggregation
- 引入多种距离度量替代传统欧氏距离，更精准评估梯度过时程度
- 在非独立同分布数据下，新度量使模型收敛更快、性能更稳定
- 适合研究异步联邦学习优化或部署实际系统的研究者参考
在异步联邦学习中，客户端根据计算速度不同时发送更新，常使用全局模型的过时版本。这种过时性会降低全局模型的收敛性和准确性。此前工作如AsyncFedED提出使用欧氏距离衡量过时程度并进行自适应聚合。本文进一步探索其他距离度量，以更准确捕捉梯度过时的影响。我们将这些度量融入聚合过程，在异构客户端和非独立同分布数据设置下评估其对收敛速度、模型性能和训练稳定性的影响。结果表明，某些度量可实现更稳健且高效的异步联邦学习训练，为实际部署提供更强基础。
原文摘要 · Abstract (English)
In asynchronous federated learning (FL), client devices send updates to a central server at varying times based on their computational speed, often using stale versions of the global model. This staleness can degrade the convergence and accuracy of the global model. Previous work, such as AsyncFedED, proposed an adaptive aggregation method using Euclidean distance to measure staleness. In this paper, we extend this approach by exploring alternative distance metrics to more accurately capture the effect of gradient staleness. We integrate these metrics into the aggregation process and evaluate their impact on convergence speed, model performance, and training stability under heterogeneous clients and non-IID data settings. Our results demonstrate that certain metrics lead to more robust and efficient asynchronous FL training, offering a stronger foundation for practical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。