解决设备动态进出下的联邦学习收敛难题,提升系统适应性与能效。
Communication-Efficient Federated Learning under Dynamic Device Arrival and Departure: Convergence Analysis and Algorithm Design
- 基于梯度相似性加权历史模型,实现设备变动时快速初始化。
- 实验显示收敛速度提升一个数量级以上,显著降低能耗。
- 适用于真实移动场景,可无缝集成现有联邦学习框架。
大多数联邦学习(FL)方法假设设备集合固定,但实际场景中设备常因用户移动或网络切换而动态加入或离开。这种动态性带来双重挑战:一是优化目标随活跃设备变化,不同于传统静态目标;二是当前全局模型可能不再适合作为后续训练的初始化,影响适应速度、延缓收敛并降低资源效率。为此,本文首次在动态设备集下提供收敛性分析,考虑了梯度噪声、本地训练轮数和数据异质性等关键因素。基于该分析,提出一种模型初始化算法,当设备加入或离开时能快速适应。核心思想是根据梯度相似性计算历史全局模型的加权平均,优先选择与当前设备数据分布相近的模型,从而在更少轮次内恢复对分布偏移的适应能力。该算法为即插即用设计,可兼容现有联邦学习方法。实验表明,相比基线方法,本方法通常实现收敛速度提升一个数量级或更多,显著降低达到目标精度所需的能耗。
原文摘要 · Abstract (English)
Most federated learning (FL) approaches assume a fixed device set. However, real-world scenarios often involve devices dynamically joining or leaving the system, driven by, e.g., user mobility patterns or handovers across cell boundaries. This dynamic setting introduces unique challenges: (1) the optimization objective evolves with the active device set, unlike traditional FL's static objective; and (2) the current global model may no longer serve as an effective initialization for subsequent rounds, potentially hindering adaptation, delaying convergence, and reducing resource efficiency. To address these challenges, we first provide a convergence analysis for FL under a dynamic device set, accounting for factors such as gradient noise, local training iterations, and data heterogeneity in this practical setting. Motivated by this analysis, we propose a model initialization algorithm that enables rapid adaptation whenever devices join or leave the network. Our key idea is to compute a weighted average of previous global models, guided by gradient similarity, to prioritize models trained on data distributions that closely align with the current device set, thereby accelerating recovery from distribution shifts in fewer training rounds. This plug-and-play algorithm is designed to integrate seamlessly with existing FL methods, offering broad applicability. Experiments demonstrate that our approach achieves convergence speedups typically an order of magnitude or more compared to baselines, which we show drastically reduces energy consumption to reach a target accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。