通过参数敏感性评估模型过时程度，提升异步联邦学习性能
FedPSA: Modeling Behavioral Staleness in Asynchronous Federated Learning
- 用参数敏感性衡量模型过时，比传统方法更精细
- 动态调整对旧信息的容忍度，实验显示最高提升6.37%
- 适合追求高效训练的分布式学习研究者
异步联邦学习（AFL）通过不等待慢速客户端实现并行训练，显著提升训练速度。然而，异步带来的模型过时问题会降低性能。现有方法仅以当前模型与全局模型的轮次差作为过时度量，过于粗略且忽略模型自身状态，限制了性能上限。本文提出FedPSA（基于参数敏感性的异步联邦学习），利用参数敏感性量化模型过时程度，并构建动态动量队列实时判断训练阶段，从而动态调节对过时信息的容忍度。在多个数据集上的大量实验表明，FedPSA性能显著优于基线方法，最高提升6.37%，较当前最优方法提升1.93%。
原文摘要 · Abstract (English)
Asynchronous Federated Learning (AFL) has emerged as a significant research area in recent years. By not waiting for slower clients and executing the training process concurrently, it achieves faster training speed compared to traditional federated learning. However, due to the staleness introduced by the asynchronous process, its performance may degrade in some scenarios. Existing methods often use the round difference between the current model and the global model as the sole measure of staleness, which is coarse-grained and lacks observation of the model itself, thereby limiting the performance ceiling of asynchronous methods. In this paper, we propose FedPSA (Parameter Sensitivity-based Asynchronous Federated Learning), a more fine-grained AFL framework that leverages parameter sensitivity to measure model obsolescence and establishes a dynamic momentum queue to assess the current training phase in real time, thereby adjusting the tolerance for outdated information dynamically. Extensive experiments on multiple datasets and comparisons with various methods demonstrate the superior performance of FedPSA, achieving up to 6.37\% improvement over baseline methods and 1.93\% over the current state-of-the-art method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。