arXiv:2503.15111cs.LG2025-03ICLR被引 9

提出分层自适应权重压缩,提升联邦学习模型泛化能力

FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinking

  • 按层自适应调整权重压缩因子,避免使用代理数据
  • 压缩因子总和小于1时,模型泛化性能显著提升
  • 兼容现有方法,适合注重隐私与性能的联邦学习场景

在联邦学习中,通常对本地模型进行加权聚合以生成全局模型,且聚合权重之和为1。近期研究发现,当权重总和(即压缩因子)小于1时,全局模型的泛化性能更优,但如何有效学习该因子仍缺乏系统性方法,尤其在考虑隐私保护与各层差异方面。为此,本文提出一种新型模型聚合策略——分层自适应权重压缩(FedLWS),在层间自适应地设计压缩因子,无需在代理数据上优化。我们首先分析了训练过程中影响压缩因子的因素,并基于全局模型各层的差异计算分层压缩因子。该方法具有良好的灵活性,可轻松集成至多种现有联邦学习框架。在多种场景下的大量实验表明,其性能优于多个先进方法,为提升联邦学习全局模型效果提供了一种有前景的工具。

原文摘要 · Abstract (English)

In Federated Learning (FL), weighted aggregation of local models is conducted to generate a new global model, and the aggregation weights are typically normalized to 1. A recent study identifies the global weight shrinking effect in FL, indicating an enhancement in the global model's generalization when the sum of weights (i.e., the shrinking factor) is smaller than 1, where how to learn the shrinking factor becomes crucial. However, principled approaches to this solution have not been carefully studied from the adequate consideration of privacy concerns and layer-wise distinctions. To this end, we propose a novel model aggregation strategy, Federated Learning with Adaptive Layer-wise Weight Shrinking (FedLWS), which adaptively designs the shrinking factor in a layer-wise manner and avoids optimizing the shrinking factors on a proxy dataset. We initially explored the factors affecting the shrinking factor during the training process. Then we calculate the layer-wise shrinking factors by considering the distinctions among each layer of the global model. FedLWS can be easily incorporated with various existing methods due to its flexibility. Extensive experiments under diverse scenarios demonstrate the superiority of our method over several state-of-the-art approaches, providing a promising tool for enhancing the global model in FL.

联邦学习模型聚合权重压缩隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。