动态调整节点连接与权重，让分布式训练更高效。
Cooperative SGD with Dynamic Mixing Matrices
- 设计可变拓扑和非均匀加权的协同优化框架
- 理论证明收敛性优于或匹配现有方法
- 适合边缘计算、异构设备的分布式训练场景
当前主流的机器学习训练方法是随机梯度下降（SGD）。在分布式环境下，已有理论表明特定条件下基于SGD的算法可收敛。然而，大量现有工作假设设备间连接结构固定且各节点贡献均等，实验表明此假设次优。通过动态调整网络拓扑和客户端选择，并采用非均匀聚合策略，可显著提升模型性能。本文提出一个统一框架，涵盖多种基于局部更新的分布式SGD算法，支持动态拓扑，并在理论上提供优于或匹配现有工作的收敛性保证。
原文摘要 · Abstract (English)
One of the most common methods to train machine learning algorithms today is the stochastic gradient descent (SGD). In a distributed setting, SGD-based algorithms have been shown to converge theoretically under specific circumstances. A substantial number of works in the distributed SGD setting assume a fixed topology for the edge devices. These papers also assume that the contribution of nodes to the global model is uniform. However, experiments have shown that such assumptions are suboptimal and a non uniform aggregation strategy coupled with a dynamically shifting topology and client selection can significantly improve the performance of such models. This paper details a unified framework that covers several Local-Update SGD-based distributed algorithms with dynamic topologies and provides improved or matching theoretical guarantees on convergence compared to existing work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。