针对边缘计算中设备异构问题,提出自适应协作的去中心化学习算法。
Chisme: Heterogeneity-Aware Gossip Learning
- 根据模型交换推断客户端数据分布相似性,动态调整融合权重。
- 在不同网络条件下,收敛更快、最终损失更低、客户端性能差异更小。
- 适合资源受限、连接不稳定且数据分布不均的边缘智能场景。
随着终端设备能力提升和边缘智能服务需求增长,分布式学习成为边缘智能的关键技术。现有联邦学习(FL)与去中心化联邦学习(DFL)虽能保护隐私,但大多假设数据分布同质,难以应对真实边缘环境中的异构性。本文提出Chisme,一种全新的完全去中心化分布式学习算法,旨在解决边缘场景下数据分布异构、连接间歇性及网络基础设施稀疏等问题。Chisme通过分析客户端间模型交换所体现的数据分布亲和性,动态决定接收模型对本地模型的影响力,从而实现泛化知识与特定知识间的策略平衡。我们在图像识别和时间序列预测任务中评估Chisme,涵盖多种网络连通性条件,结果表明其在几乎所有情况下均优于当前先进方法:客户端训练收敛更快、最终损失更低、跨客户端性能差异更小。
原文摘要 · Abstract (English)
As end-user device capability increases and demand for intelligent services at the Internet's edge rises, distributed learning has emerged as a key enabling technology for the intelligent edge. Existing approaches like federated learning (FL) and decentralized FL (DFL) enable privacy-preserving distributed learning among clients, while gossip learning (GL) approaches have emerged to address the potential challenges in resource-constrained, connectivity-challenged infrastructure-less environments. However, most distributed learning approaches assume largely homogeneous data distributions and may not consider or exploit the heterogeneity of clients and their underlying data distributions. This paper introduces Chisme, a novel fully decentralized distributed learning algorithm designed to address the challenges of implementing robust intelligence in network edge contexts characterized by heterogeneous data distributions, episodic connectivity, and sparse network infrastructure or lack thereof. Chisme leverages the affinity between clients' underlying data distributions calculated from received model exchanges to inform how much influence received models have when merging into the local model. By doing so, it enables clients to strategically balance between broader collaboration to build more general knowledge and more selective collaboration to build specific knowledge. We evaluate Chisme against contemporary approaches using image recognition and time-series prediction scenarios while considering different network connectivity conditions, representative of real-world distributed intelligent systems running at the network's edge. Our experiments demonstrate that Chisme outperforms state-of-the-art edge intelligence approaches in almost every case -- clients using Chisme exhibit faster training convergence, lower final loss after training, and lower performance disparity between clients.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。