用粗量化降低异步ADMM通信开销，提升大规模分布式学习效率
Communication-Efficient Distributed Asynchronous ADMM
- 在异步ADMM中对交换数据进行粗量化，减少通信负担
- 实验验证在神经网络等任务上仍能保持收敛性
- 适合通信资源受限的联邦学习与分布式优化场景
在分布式优化和联邦学习中，异步交替方向乘子法（ADMM）是解决大规模优化、数据隐私、慢节点及多样目标函数的有力工具。然而，当节点通信预算有限或需传输的数据量过大时，通信成本可能成为主要瓶颈。本文提出在异步ADMM的通信数据中引入粗量化策略，以降低大规模联邦学习与分布式优化应用中的通信开销。通过多个分布式学习任务的实验验证，该方法在神经网络等场景下仍可保证收敛性。
原文摘要 · Abstract (English)
In distributed optimization and federated learning, asynchronous alternating direction method of multipliers (ADMM) serves as an attractive option for large-scale optimization, data privacy, straggler nodes and variety of objective functions. However, communication costs can become a major bottleneck when the nodes have limited communication budgets or when the data to be communicated is prohibitively large. In this work, we propose introducing coarse quantization to the data to be exchanged in aynchronous ADMM so as to reduce communication overhead for large-scale federated learning and distributed optimization applications. We experimentally verify the convergence of the proposed method for several distributed learning tasks, including neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。