arXiv:2602.03383cs.LGcs.DC2026-02中稿 · publication in the…

动态调整通信拓扑,提升非独立同分布数据下的去中心化学习效果。

Dynamic Topology Optimization for Non-IID Data in Decentralized Learning

  • 基于模型差异选择通信伙伴,动态优化网络结构。
  • 在CIFAR-10上比最优基线高1.12倍测试准确率。
  • 无需全局信息,适合隐私敏感、节点多的分布式场景。

去中心化学习(DL)允许多个节点在无中央协调下协作训练模型,有利于隐私保护与可扩展性。然而,当数据分布非独立同分布(non-IID)且通信拓扑固定时,性能显著下降。为此,我们提出Morph算法,通过最大模型差异自适应选择通信伙伴,在保持固定入度的前提下,利用基于播送的节点发现和多样性驱动的邻居选择,动态重构通信图,增强对数据异构性的鲁棒性。在最多100个节点的CIFAR-10和FEMNIST实验中,Morph始终优于静态和流行病式基线,接近全连接上界。在CIFAR-10上,测试准确率相对最优基线提升1.12倍;在FEMNIST上,准确率高出流行病学习1.08倍。50节点部署时,其与全连接上界的差距缩小至0.5个百分点以内。结果表明,Morph实现更高最终精度、更快收敛与更稳定学习(节点间方差更低),同时减少通信轮次,且无需全局知识。

原文摘要 · Abstract (English)

Decentralized learning (DL) enables a set of nodes to train a model collaboratively without central coordination, offering benefits for privacy and scalability. However, DL struggles to train a high accuracy model when the data distribution is non-independent and identically distributed (non-IID) and when the communication topology is static. To address these issues, we propose Morph, a topology optimization algorithm for DL. In Morph, nodes adaptively choose peers for model exchange based on maximum model dissimilarity. Morph maintains a fixed in-degree while dynamically reshaping the communication graph through gossip-based peer discovery and diversity-driven neighbor selection, thereby improving robustness to data heterogeneity. Experiments on CIFAR-10 and FEMNIST with up to 100 nodes show that Morph consistently outperforms static and epidemic baselines, while closely tracking the fully connected upper bound. On CIFAR-10, Morph achieves a relative improvement of 1.12x in test accuracy compared to the state-of-the-art baselines. On FEMNIST, Morph achieves an accuracy that is 1.08x higher than Epidemic Learning. Similar trends hold for 50 node deployments, where Morph narrows the gap to the fully connected upper bound within 0.5 percentage points on CIFAR-10. These results demonstrate that Morph achieves higher final accuracy, faster convergence, and more stable learning as quantified by lower inter-node variance, while requiring fewer communication rounds than baselines and no global knowledge.

去中心化学习非IID拓扑优化联邦学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。