arXiv:2606.06687cs.LGcs.DC2026-06

无需服务器的联邦学习新框架,提升异构优化器下的训练效率。

Towards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers

  • 通过设备间一次性初始化形成集群,实现无服务器的半去中心化训练。
  • 引入有效损失函数与图正则化,保障不同优化器下全局收敛与共识。
  • 基于切赫不等式迭代聚类,适配异构数据与优化器,适合资源受限场景。

我们研究了在具有异构机器学习优化器的去中心化联邦学习(FL)中,集群形成问题,包括集群数量与组成。尽管集中式联邦学习中的聚类已实现可扩展性和资源节约,但其在全去中心化环境中的价值与发展仍待探索。此类环境中优化集群形成极具挑战性,尤其因网络图结构、本地数据异质性及不同本地模型优化器之间的复杂耦合所致。为此,我们提出无服务器半去中心化联邦学习(SSD-FL),无需持续服务器基础设施。在SSD-FL中,集群通过轻量级的一次性设备到设备(D2D)初始化阶段形成,之后实际的模型训练(含共识与收敛过程)完全无服务器化。功能上,SSD-FL将全局轮次划分为簇内与簇间阶段,通过结合设备特定优化器与基于网络图的正则化的新型“有效损失函数”,确保全局收敛与共识。接着,利用切赫不等式中的共识间隙,开发出一种迭代聚类算法,并基于推导出的收敛与共识边界进行评估,其中包含独特评分指标以量化设备间的数据与优化器异质性。最后,对三类去中心化联邦学习方法的实验评估表明,SSD-FL在多种网络图、数据集和本地优化器配置下,均提升了收敛速度与通信效率。

原文摘要 · Abstract (English)

We investigate cluster formation, involving the number and composition of clusters, in decentralized federated learning (FL) with heterogeneous machine learning (ML) optimizers. While clustering in centralized FL has enabled scalability and resource savings, its value and development in fully decentralized environments have yet to be explored. Optimizing cluster formation in such environments is challenging, especially due to the complex coupling between network graph structures, local data heterogeneity, and different local ML model optimizers. To address these challenges, we propose serverless semi-decentralized FL (SSD-FL), a methodology requiring no persistent server infrastructure. In SSD-FL, cluster formation occurs via a lightweight, one-time device-to-device (D2D) initialization phase, after which actual ML model training (alongside consensus and convergence processes) is fully serverless. Functionally, SSD-FL segments global rounds into intra-cluster and inter-cluster regimes, ensuring global convergence and consensus through novel "effective loss functions" that integrate device-specific ML optimizers with network graph-based regularization. Next, SSD-FL leverages the consensus gap via the Cheeger inequality to develop an iterative clustering algorithm evaluated against our derived convergence and consensus bounds, which incorporate a unique scoring metric to quantify data and optimizer heterogeneity across devices. Finally, experimental evaluation against three categories of decentralized FL methodologies validate that SSD-FL improves both convergence speeds and communication efficiency across various network graphs, datasets, and local optimizer regimes.

联邦学习去中心化优化器异构无服务器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。