解析分布式优化中误差反馈算法的收敛性，给出最优步长与函数设计。
A Tight Theory of Error Feedback Algorithms in Distributed Optimization
- 通过构造定制化李雅普诺夫函数分析算法性能
- 确定了经典误差反馈与EF21的最优步长选择
- 适用于大规模分布式学习场景，理论严谨
通信开销是分布式学习和一阶优化中的主要瓶颈。一种常见方法是压缩代理间交换的梯度信息，但压缩通常会降低基于梯度方法的收敛保证。误差反馈机制提供了一种简单且计算成本低的解决方案，但已有多种变体，其相对性能尚不明确。本文对文献中两种主流误差反馈算法——经典误差反馈（EF）和误差反馈21（EF21）——进行了紧致的收敛性分析，通过识别最优步长选择并构造针对每种方法的最优李雅普诺夫函数，得到的结果不依赖于代理数量，并在单代理情形下恢复了已知的最佳收敛保证。
原文摘要 · Abstract (English)
Communication costs are a major bottleneck in distributed learning and first-order optimization. A common approach to alleviate this issue is to compress the gradient information exchanged between agents. However, such compression typically degrades the convergence guarantees of gradient-based methods. Error feedback mechanisms provide a simple and computationally cheap remedy for this issue, but numerous variants have been proposed, and their relative performance remains poorly understood. This paper provides tight convergence analyses for two of the main error-feedback algorithms from the literature, the classic Error Feedback method (EF) and Error Feedback 21 (EF21), by identifying optimal step-size choices and constructing optimal Lyapunov functions tailored to each method. The results hold independently of the number of agents and recover the known best guarantees possible in the single-agent regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。