用信息熵设计调度策略，显著降低去中心化学习通信开销。
Information Entropy-Based Scheduling for Communication-Efficient Decentralized Learning
- 基于信息熵构建节点与链路调度重要性度量
- 通信预算低时收敛速度提升60%，高预算下性能相当或更优
- 适合资源受限的分布式学习场景，如边缘计算
本文针对资源受限网络中的去中心化随机梯度下降（D-SGD）问题，提出基于节点和链路的调度策略以提升通信效率。在每轮D-SGD中，仅随机激活满足通信成本约束的少数不相交节点子集或链路子集。我们提出一种基于信息熵的新重要性度量，用于确定节点与链路的调度概率。通过大量模拟验证，本方法在节点调度中优于基于介数中心性（BC）的方法，在通信预算较低时实现最高60%的通信开销降低并加速收敛；在通信预算高于60%时仍保持相当或更优性能。在链路调度中，本方法结果优于或等同于MATCHA方法。
原文摘要 · Abstract (English)
This paper addresses decentralized stochastic gradient descent (D-SGD) over resource-constrained networks by introducing node-based and link-based scheduling strategies to enhance communication efficiency. In each iteration of the D-SGD algorithm, only a few disjoint subsets of nodes or links are randomly activated, subject to a given communication cost constraint. We propose a novel importance metric based on information entropy to determine node and link scheduling probabilities. We validate the effectiveness of our approach through extensive simulations, comparing it against state-of-the-art methods, including betweenness centrality (BC) for node scheduling and \textit{MATCHA} for link scheduling. The results show that our method consistently outperforms the BC-based method in the node scheduling case, achieving faster convergence with up to 60\% lower communication budgets. At higher communication budgets (above 60\%), our method maintains comparable or superior performance. In the link scheduling case, our method delivers results that are superior to or on par with those of \textit{MATCHA}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。