arXiv:2601.19938cs.LGcs.AI2026-01

通过二阶信息优化去中心化联邦学习的聚合方式,提升异构设备下的模型收敛速度。

DecHW: Heterogeneous Decentralized Federated Learning Exploiting Second-Order Information

  • 利用本地模型的二阶信息计算共识权重,动态调整邻居更新贡献。
  • 在计算机视觉任务中实现更低通信开销下的稳定模型泛化性能。
  • 适合设备异构性高、通信资源受限的分布式学习场景。

去中心化联邦学习(DFL)是一种无需服务器的协同机器学习范式,设备间直接与邻近设备交换模型信息以学习通用模型。然而,个体经验差异和设备间交互水平不同导致数据与模型初始化存在异质性,进而引发局部模型参数差异,造成收敛缓慢。本文通过显式建模局部模型间参数层面的可信度差异来缓解此类异质性。提出一种新型聚合方法,捕捉局部模型间的参数差异,并对邻域更新进行鲁棒聚合。具体而言,通过近似局部模型在其本地数据集上的二阶信息生成共识权重,用于缩放邻域更新后聚合为全局邻域表示。在多个计算机视觉任务的广泛实验中,所提方法在降低通信成本的同时展现出强健的局部模型泛化能力。

原文摘要 · Abstract (English)

Decentralized Federated Learning (DFL) is a serverless collaborative machine learning paradigm where devices collaborate directly with neighbouring devices to exchange model information for learning a generalized model. However, variations in individual experiences and different levels of device interactions lead to data and model initialization heterogeneities across devices. Such heterogeneities leave variations in local model parameters across devices that leads to slower convergence. This paper tackles the data and model heterogeneity by explicitly addressing the parameter level varying evidential credence across local models. A novel aggregation approach is introduced that captures these parameter variations in local models and performs robust aggregation of neighbourhood local updates. Specifically, consensus weights are generated via approximation of second-order information of local models on their local datasets. These weights are utilized to scale neighbourhood updates before aggregating them into global neighbourhood representation. In extensive experiments with computer vision tasks, the proposed approach shows strong generalizability of local models at reduced communication costs.

联邦学习去中心化异构性二阶信息

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。