提出新方法缓解边缘计算中慢节点问题,提升联邦学习效率。
Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach
- 通过非均衡更新机制解耦服务器与客户端进度,避免等待慢节点。
- 每轮可执行τ次本地更新,通信效率提升τ倍,收敛速度达O(√d/(τT))。
- 适合存在设备延迟的分布式系统,尤其适用于资源不均的边缘场景。
分割联邦学习(SFL)通过结合联邦学习(FL)的并行性与分割学习(SL)的计算卸载,实现了在边缘设备上的可扩展训练。然而,SFL受分布式学习中普遍存在的慢节点问题严重影响。该问题因分割服务器与客户端间的依赖关系而加剧:服务器端模型更新需等待客户端传回激活值,同步要求引入显著延迟,使慢节点成为系统可扩展性与效率的关键瓶颈。为缓解此问题,本文提出零阶优化下的鲁棒分割联邦学习算法MU-SplitFed,通过简单的非均衡更新机制,使服务器在每轮客户端通信中可执行τ次本地更新,从而解耦训练进度与慢节点延迟。该方法在非凸目标下实现O(√d/(τT))的收敛速率,通信轮数上获得τ倍线性加速。实验表明,即使存在慢节点,MU-SplitFed仍持续优于基线方法,并通过自适应调节τ有效抑制慢节点影响。代码已开源:https://github.com/Johnny-Zip/MU-SplitFed。
原文摘要 · Abstract (English)
Split Federated Learning (SFL) enables scalable training on edge devices by combining the parallelism of Federated Learning (FL) with the computational offloading of Split Learning (SL). Despite its great success, SFL suffers significantly from the well-known straggler issue in distributed learning systems. This problem is exacerbated by the dependency between Split Server and clients: the Split Server side model update relies on receiving activations from clients. Such synchronization requirement introduces significant time latency, making straggler a critical bottleneck to the scalability and efficiency of the system. To mitigate this problem, we propose MU-SplitFed, a straggler-resilient SFL algorithm in zeroth-order optimization that decouples training progress from straggler delays via a simple yet effective unbalanced update mechanism. By enabling the server to perform $τ$ local updates per client round, MU-SplitFed achieves a convergence rate of $O(\sqrt{d/(τT)})$ for non-convex objectives, demonstrating a linear speedup of $τ$ in communication rounds. Experiments demonstrate that MU-SplitFed consistently outperforms baseline methods with the presence of stragglers and effectively mitigates their impact through adaptive tuning of $τ$. The code for this project is available at https://github.com/Johnny-Zip/MU-SplitFed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。