提出自适应学习率机制，提升联邦学习收敛速度与稳定性。
FedDuA: Doubly Adaptive Federated Learning
- 基于镜面下降视角，双适应调节全局学习率
- 在多种设置下超越基线，且对超参不敏感
- 无需额外通信开销，适合实际部署
联邦学习是一种分布式学习框架，客户端在不共享原始数据的情况下协作训练全局模型。虽然FedAvg是主流算法，但其常因本地数据异质性和参数空间各向异性导致收敛缓慢。本文从镜面下降角度形式化中央服务器优化过程，提出新框架FedDuA，通过同时考虑客户端间和坐标级的局部更新异质性，自适应选择全局学习率。证明所提双重自适应步长规则为极小极大最优，并给出凸目标下的收敛分析。该方法在客户端无需额外通信或计算成本的前提下，大量数值实验表明其在多种场景下优于基线，且对超参数选择具有鲁棒性。
原文摘要 · Abstract (English)
Federated learning is a distributed learning framework where clients collaboratively train a global model without sharing their raw data. FedAvg is a popular algorithm for federated learning, but it often suffers from slow convergence due to the heterogeneity of local datasets and anisotropy in the parameter space. In this work, we formalize the central server optimization procedure through the lens of mirror descent and propose a novel framework, called FedDuA, which adaptively selects the global learning rate based on both inter-client and coordinate-wise heterogeneity in the local updates. We prove that our proposed doubly adaptive step-size rule is minimax optimal and provide a convergence analysis for convex objectives. Although the proposed method does not require additional communication or computational cost on clients, extensive numerical experiments show that our proposed framework outperforms baselines in various settings and is robust to the choice of hyperparameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。