arXiv:2503.00378cs.LGcs.AI2025-03被引 1

用本地统计量指导模型,实现高效私密的异构联邦学习。

Conditioning on Local Statistics for Scalable Heterogeneous Federated Learning

  • 各客户端计算本地数据统计量(均值、协方差等)作为条件信号。
  • 在异构数据下显著提升模型收敛性与性能,且无需通信额外信息。
  • 适合注重隐私与可扩展性的实际联邦学习场景。

联邦学习是一种分布式机器学习方法,多个客户端在不共享本地数据的前提下协同训练模型,有助于保护隐私。其挑战在于客户端间存在异构的数据分布,导致全局模型难以有效泛化。本文提出利用本地特征统计量——即各客户端仅基于本地训练数据独立计算的统计特性(如均值、协方差和高阶矩),来捕捉本地数据分布特征。这些统计量不向其他客户端或中心节点共享。训练时,它们帮助模型学习对本地分布的条件适应;推理时,则引导客户端的预测结果。实验表明,该方法能有效应对联邦中的异构数据,相比需识别同分布节点的方法具有更好可扩展性,同时保持隐私,无需额外通信。

原文摘要 · Abstract (English)

Federated learning is a distributed machine learning approach where multiple clients collaboratively train a model without sharing their local data, which contributes to preserving privacy. A challenge in federated learning is managing heterogeneous data distributions across clients, which can hinder model convergence and performance due to the need for the global model to generalize well across diverse local datasets. We propose to use local characteristic statistics, by which we mean some statistical properties calculated independently by each client using only their local training dataset. These statistics, such as means, covariances, and higher moments, are used to capture the characteristics of the local data distribution. They are not shared with other clients or a central node. During training, these local statistics help the model learn how to condition on the local data distribution, and during inference, they guide the client's predictions. Our experiments show that this approach allows for efficient handling of heterogeneous data across the federation, has favorable scaling compared to approaches that directly try to identify peer nodes that share distribution characteristics, and maintains privacy as no additional information needs to be communicated.

联邦学习异构数据隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。