arXiv:2410.22192cs.LGcs.IT2024-10被引 2

用参数更新年龄优化通信,提升联邦学习效率

$r$Age-$k$: Communication-Efficient Federated Learning Using Age Factor

  • 根据参数更新频率动态选择客户端梯度
  • 在非独立同分布数据下训练速度更快
  • 适合资源受限的分布式场景

联邦学习(FL)中,多个客户端由参数服务器(PS)协调训练统一模型,但面临数据异质性和通信开销两大挑战。本文提出基于信息年龄指标的通信高效算法,引入年龄向量记录各参数被客户端更新的频次。PS据此选择性请求特定梯度索引更新,并利用年龄向量识别数据分布相似的客户端,将其聚类后协同调度组内梯度更新。在非独立同分布的MNIST和CIFAR10数据集上实验表明,该方法显著加速训练,优于现有通信高效策略。

原文摘要 · Abstract (English)

Federated learning (FL) is a collaborative approach where multiple clients, coordinated by a parameter server (PS), train a unified machine-learning model. The approach, however, suffers from two key challenges: data heterogeneity and communication overhead. Data heterogeneity refers to inconsistencies in model training arising from heterogeneous data at different clients. Communication overhead arises from the large volumes of parameter updates exchanged between the PS and clients. Existing solutions typically address these challenges separately. This paper introduces a new communication-efficient algorithm that uses the age of information metric to simultaneously tackle both limitations of FL. We introduce age vectors at the PS, which keep track of how often the different model parameters are updated from the clients. The PS uses this to selectively request updates for specific gradient indices from each client. Further, the PS employs age vectors to identify clients with statistically similar data and group them into clusters. The PS combines the age vectors of the clustered clients to efficiently coordinate gradient index updates among clients within a cluster. We evaluate our approach using the MNIST and CIFAR10 datasets in highly non-i.i.d. settings. The experimental results show that our proposed method can expedite training, surpassing other communication-efficient strategies in efficiency.

联邦学习通信效率非iid

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。