提出一种高效分布式学习算法，兼顾计算与通信开销。
Jointly Computation- and Communication-Efficient Distributed Learning
- 基于ADMM框架，局部训练使用随机梯度提升效率
- 通信间隔内执行多轮训练，传输数据进行压缩
- 理论证明强凸条件下线性收敛，适合大规模分布式场景
针对无向网络中的分布式学习问题，本文设计了一种新型基于ADMM的算法，实现计算与通信双重高效。通过在局部训练中采用随机梯度保证计算效率；同时，通过在通信轮次间执行多轮训练、使用压缩传输来提升通信效率。理论证明该算法在强凸设定下具有精确线性收敛性。数值实验在分类任务上对比现有最优方法，验证了其理论优势。
原文摘要 · Abstract (English)
We address distributed learning problems over undirected networks. Specifically, we focus on designing a novel ADMM-based algorithm that is jointly computation- and communication-efficient. Our design guarantees computational efficiency by allowing agents to use stochastic gradients during local training. Moreover, communication efficiency is achieved as follows: i) the agents perform multiple training epochs between communication rounds, and ii) compressed transmissions are used. We prove exact linear convergence of the algorithm in the strongly convex setting. We corroborate our theoretical results by numerical comparisons with state of the art techniques on a classification task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。