解决联邦学习中客户端参与不全导致的误差问题。
FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation
- 利用最新存储的客户端更新，模拟缺席客户端的贡献。
- 在非凸条件下证明可消除部分参与误差，实验表现优于现有方法。
- 支持量化存储，内存减少50%~87.5%，适合资源受限场景。
联邦学习因数据异构性面临严峻挑战，导致梯度噪声、客户端漂移及部分客户端参与误差，其中后者最为普遍却未被充分研究。本文提出FedAdaVR，一种结合自适应优化器与方差缩减技术的新算法，通过利用当前训练轮次中仍存有客户端的最近更新，即使其缺席也模拟其参与。此外，提出FedAdaVR-Quant，以量化形式存储客户端更新，使内存需求降低50%、75%和87.5%，同时保持优异模型性能。我们在一般非凸条件下分析了FedAdaVR的收敛性，证明该算法可消除部分客户端参与误差。在多个数据集上，无论独立同分布（IID）或非独立同分布（non-IID）设置下，广泛实验表明其始终优于现有先进方法。
原文摘要 · Abstract (English)
Federated learning (FL) encounters substantial challenges due to heterogeneity, leading to gradient noise, client drift, and partial client participation errors, the last of which is the most pervasive but remains insufficiently addressed in current literature. In this paper, we propose FedAdaVR, a novel FL algorithm aimed at solving heterogeneity issues caused by sporadic client participation by incorporating an adaptive optimiser with a variance reduction technique. This method takes advantage of the most recent stored updates from clients, even when they are absent from the current training round, thereby emulating their presence. Furthermore, we propose FedAdaVR-Quant, which stores client updates in quantised form, significantly reducing the memory requirements (by 50%, 75%, and 87.5%) of FedAdaVR while maintaining highly competitive model performance. We analyse the convergence behaviour of FedAdaVR under general nonconvex conditions and prove that our proposed algorithm can eliminate partial client participation error. Extensive experiments conducted on multiple datasets, under both independent and identically distributed (IID) and non-IID settings, demonstrate that FedAdaVR consistently outperforms state-of-the-art baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。