为误差反馈方法提供精确理论分析，揭示其最优收敛速率。
Tight analyses of first-order methods with error feedback
- 构建李雅普诺夫函数，推导出误差反馈的最优收敛率
- 证明了EF和EF²¹方法的下界，与上界匹配
- 适用于研究分布式优化中通信压缩机制的学者
在分布式学习中，通信常成为主要计算瓶颈。常用缓解策略是压缩交换信息以降低通信开销。为抵消压缩带来的收敛性能下降，引入了误差反馈机制（如EF和EF²¹）。本文对这两种方法进行了紧致分析，找到使每种方法收敛速率最优的李雅普诺夫函数，并给出匹配的下界。该严谨方法提供了精确性能保证，实现了EF、EF²¹与压缩梯度下降之间的公平对比。分析基于简化单智能体设定，便于获得清晰的理论洞见。
原文摘要 · Abstract (English)
Communication between agents often constitutes a major computational bottleneck in distributed learning. One of the most common mitigation strategies is to compress the information exchanged, thereby reducing communication overhead. To counteract the degradation in convergence associated with compressed communication, error feedback schemes -- most notably $\mathrm{EF}$ and $\mathrm{EF}^{21}$ -- were introduced. In this work, we provide a tight analysis of both of these methods. Specifically, we find the Lyapunov function that yields the best possible convergence rate for each method -- with matching lower bounds. This principled approach yields sharp performance guarantees and enables a rigorous, apples-to-apples comparison between $\mathrm{EF}$, $\mathrm{EF}^{21}$, and compressed gradient descent. Our analysis is carried out in the simplified single-agent setting, which allows for clean theoretical insights and fair comparison of the underlying mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。