提出新型联邦学习算法，显著降低通信开销并加快收敛。
Non-Convex Optimization in Federated Learning via Variance Reduction and Adaptive Learning
- 结合动量与自适应学习率，减少梯度方差影响。
- 通信复杂度降至O(ε⁻¹)，比之前方法快一倍。
- 适合数据异构场景，缓解客户端漂移问题。
本文提出一种新型联邦学习算法，通过基于动量的方差缩减与自适应学习率，应对异构数据下的非凸优化问题。旨在降低通信与计算开销，构建可持续的联邦学习系统。针对梯度方差大和学习率调整导致收敛慢的问题，实验在异构数据的图像分类任务中验证了算法有效性：达到ε-平稳点的通信复杂度为O(ε⁻¹)，优于多数先前工作的O(ε⁻²)。所提算法在收敛速度、通信轮次与测试准确率间保持良好平衡，有效缓解异构场景下的客户端漂移。实验在MNIST、CIFAR-10数据集上均表现出高效性。
原文摘要 · Abstract (English)
This paper proposes a novel federated algorithm that leverages momentum-based variance reduction with adaptive learning to address non-convex settings across heterogeneous data. We intend to minimize communication and computation overhead, thereby fostering a sustainable federated learning system. We aim to overcome challenges related to gradient variance, which hinders the model's efficiency, and the slow convergence resulting from learning rate adjustments with heterogeneous data. The experimental results on the image classification tasks with heterogeneous data reveal the effectiveness of our suggested algorithms in non-convex settings with an improved communication complexity of $\mathcal{O}(ε^{-1})$ to converge to an $ε$-stationary point - compared to the existing communication complexity $\mathcal{O}(ε^{-2})$ of most prior works. The proposed federated version maintains the trade-off between the convergence rate, number of communication rounds, and test accuracy while mitigating the client drift in heterogeneous settings. The experimental results demonstrate the efficiency of our algorithms in image classification tasks (MNIST, CIFAR-10) with heterogeneous data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。