提出轻量级自适应联邦优化算法，提升通信与内存效率。
Efficient Adaptive Federated Optimization
- 服务器不传预条件矩阵，降低通信开销
- 客户端用低内存自适应优化器，节省设备资源
- 理论证明收敛速度与复杂方法相当，适合大规模设备部署
自适应优化在联邦学习中至关重要，同时在服务器和客户端实现自适应对性能提升尤为关键。然而，联合自适应系统常因通信和内存资源限制而难以扩展。本文提出一类高效自适应算法：FedAda²及其增强版FedAda²++，专为大规模跨设备联邦环境设计。FedAda²通过避免在服务器与客户端间传输预条件矩阵，优化了通信效率；FedAda²++进一步在客户端引入内存高效的自适应优化器，显著降低设备内存占用。理论上，我们证明了FedAda²和FedAda²++在一般非凸目标下的收敛速率，与资源消耗更大的联合自适应方法相同。在图像和文本数据集上的大量实验验证了联合自适应的优势及FedAda²/FedAda²++的有效性。
原文摘要 · Abstract (English)
Adaptive optimization is critical in federated learning, where enabling adaptivity on both the server and client sides has proven essential for achieving optimal performance. However, the scalability of such jointly adaptive systems is often hindered by resource limitations in communication and memory. In this paper, we introduce a class of efficient adaptive algorithms, named $FedAda^2$ and its enhanced version $FedAda^2$++, designed specifically for large-scale, cross-device federated environments. $FedAda^2$ optimizes communication efficiency by avoiding the transfer of preconditioners between the server and clients. Additionally, $FedAda^2$++ extends this approach by incorporating memory-efficient adaptive optimizers on the client side, further reducing on-device memory usage. Theoretically, we demonstrate that $FedAda^2$ and $FedAda^2$++ achieve the same convergence rates for general, non-convex objectives as its more resource-intensive counterparts that directly integrate joint adaptivity. Extensive empirical evaluations on image and text datasets demonstrate both the advantages of joint adaptivity and the effectiveness of $FedAda^2$/$FedAda^2$++.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。