提出新算法LIPPAX,提升联邦变分不等式求解速度与稳定性。
Faster Rates For Federated Variational Inequalities
- 改进分析框架,证明局部额外梯度法在平滑单调问题上收敛更快。
- 新算法LIPPAX有效抑制客户端漂移,在多种条件下实现更优收敛率。
- 适用于多类联邦学习场景,尤其适合高方差或非凸问题的优化者。
本文研究联邦学习中求解随机变分不等式(VIs)的问题。尽管已有进展,但现有收敛速率仍落后于联邦凸优化的最优界。为此,我们通过精细化分析,证明经典局部额外梯度(Local Extra SGD)算法在一般光滑单调变分不等式下可获得更紧的收敛保证。同时,我们发现该算法存在客户端过度漂移的固有缺陷。为此,提出新算法——带额外步的局部不精确邻近点算法(LIPPAX),能有效缓解漂移问题,并在有界赫斯蒂安、有界算子和低方差等不同设置下实现更优收敛性能。最后,我们将结果扩展至联邦复合变分不等式,建立了新的收敛保障。
原文摘要 · Abstract (English)
In this paper, we study federated optimization for solving stochastic variational inequalities (VIs), a problem that has attracted growing attention in recent years. Despite substantial progress, a significant gap remains between existing convergence rates and the state-of-the-art bounds known for federated convex optimization. In this work, we address this limitation by establishing a series of improved convergence rates. First, we show that, for general smooth and monotone variational inequalities, the classical Local Extra SGD algorithm admits tighter guarantees under a refined analysis. Next, we identify an inherent limitation of Local Extra SGD, which can lead to excessive client drift. Motivated by this observation, we propose a new algorithm, the Local Inexact Proximal Point Algorithm with Extra Step (LIPPAX), and show that it mitigates client drift and achieves improved guarantees in several regimes, including bounded Hessian, bounded operator, and low-variance settings. Finally, we extend our results to federated composite variational inequalities and establish improved convergence guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。