arXiv:2604.09276cs.LG2026-04被引 1

压缩通信下的分布式在线优化,实现最优后悔值并解决误差累积问题。

Distributed Online Convex Optimization with Compressed Communication: Optimal Regret and Applications

  • 将误差反馈引入正则化领袖追踪框架,缓解压缩与投影误差耦合。
  • 在凸和强凸损失下分别达到 $O(δ^{-1/2} oot{T})$ 与 $O(δ^{-1} ext{log} T)$ 的最优后悔界。
  • 适用于大规模在线学习场景,尤其适合通信受限的分布式系统。

分布式在线凸优化(D-OCO)是建模流式数据分布式场景的强大范式,但大规模应用中本地学习器与中心服务器间的通信开销巨大。为缓解此瓶颈,本文首次研究带压缩通信的D-OCO。首先,针对凸与强凸损失函数,分别建立 $Ω(δ^{-1/2} oot{T})$ 与 $Ω(δ^{-1} ext{log} T)$ 的下界,其中 $δ∈(0,1]$ 为压缩比。其次,提出一种最优算法,在凸与强凸情形下分别实现 $O(δ^{-1/2} oot{T})$ 与 $O(δ^{-1} ext{log} T)$ 的后悔界。方法通过将误差反馈机制融入Follow-the-Regularized-Leader框架,处理压缩误差与投影误差的耦合;同时采用在线压缩策略,抑制双向压缩带来的累积误差。该方法具有广泛适用性,可通过在线到批量转换推广至离线随机设置,为带压缩通信与域约束的分布式非光滑优化提供首个收敛率保证:凸函数下为 $O(δ^{-1/2}T^{-1/2})$,强凸函数下为 $O(δ^{-1}T^{-1})$。

原文摘要 · Abstract (English)

Distributed online convex optimization (D-OCO) is a powerful paradigm for modeling distributed scenarios with streaming data. However, the communication cost between local learners and the central server is substantial in large-scale applications. To alleviate this bottleneck, we initiate the study of D-OCO with compressed communication. Firstly, to quantify the compression impact, we establish the $Ω(δ^{-1/2}\sqrt{T})$ and $Ω(δ^{-1}\log{T})$ lower bounds for convex and strongly convex loss functions, respectively, where $δ\in (0,1]$ is the compression ratio. Secondly, we propose an optimal algorithm, which enjoys regret bounds of $O(δ^{-1/2}\sqrt{T})$ and $O(δ^{-1} \log T)$ for convex and strongly convex loss functions, respectively. Our method incorporates the error feedback mechanism into the Follow-the-Regularized-Leader framework to address the coupling between the compression error and the projection error. Furthermore, we employ the online compression strategy to mitigate the accumulated error arising from the bidirectional compression. Our online method has great generality, and can be extended to the offline stochastic setting via online-to-batch conversion. We establish convergence rates of $O(δ^{-1/2}T^{-1/2})$ and $O(δ^{-1} T^{-1})$ for convex and strongly convex loss functions, respectively, providing the first guarantees for distributed non-smooth optimization with compressed communication and domain constraints.

分布式优化压缩通信在线学习后悔界

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。