提出低维子空间算法FedSub，解决异构数据下联邦学习的通信与计算瓶颈。
An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
- 用低维子空间投影限制本地更新，降低通信与内存开销
- 引入低维对偶变量缓解客户端漂移问题
- 理论分析揭示步长与投影矩阵对收敛的影响，适合大规模异构场景
本文针对大规模深度神经网络在联邦学习中面临的挑战，特别是因客户端数据异构导致的客户端漂移问题以及高通信、计算和内存成本，提出FedSub——一种面向异构数据的高效子空间联邦学习算法。FedSub通过子空间投影，确保每个客户端的本地更新局限于低维子空间，从而降低通信、计算与内存消耗；同时引入低维对偶变量以缓解客户端漂移。我们提供了收敛性分析，揭示了步长与子空间投影矩阵对收敛性能的影响。实验结果表明该方法具有显著效率优势。
原文摘要 · Abstract (English)
This work addresses the key challenges of applying federated learning to large-scale deep neural networks, particularly the issue of client drift due to data heterogeneity across clients and the high costs of communication, computation, and memory. We propose FedSub, an efficient subspace algorithm for federated learning on heterogeneous data. Specifically, FedSub utilizes subspace projection to guarantee local updates of each client within low-dimensional subspaces, thereby reducing communication, computation, and memory costs. Additionally, it incorporates low-dimensional dual variables to mitigate client drift. We provide convergence analysis that reveals the impact of key factors such as step size and subspace projection matrices on convergence. Experimental results demonstrate its efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。