解决联邦学习中的标签分布偏移问题,提升目标域性能。
Overcoming label shift with target-aware federated learning
- 基于服务器端标签分布信息,动态调整模型聚合策略。
- 在图像分类任务中,相比基线方法显著提升目标域准确率。
- 特别适合客户端标签分布与目标域差异大的场景。
联邦学习允许多方在不共享私有数据的前提下协同训练模型。现有算法在目标域与客户端数据分布一致时表现良好,但实践中常出现标签分布偏移——即客户端与目标域的标签分布不同。我们实证发现这会严重降低模型性能。为此,提出FedPALS,一种基于目标域标签分布信息的可证明且实用的模型聚合方案。该方法在联邦随机梯度下降下保证无偏更新,从而在标签分布各异的客户端上实现稳健泛化。大量图像分类实验表明,FedPALS始终优于基线方法,能有效对齐模型聚合与目标域。研究还揭示:当客户端标签极度稀疏时,传统联邦学习性能大幅下降,凸显了目标感知聚合的重要性。
原文摘要 · Abstract (English)
Federated learning enables multiple actors to collaboratively train models without sharing private data. Existing algorithms are successful and well-justified in this task when the intended target domain, where the trained model will be used, shares data distribution with the aggregate of clients, but this is often violated in practice. A common reason is label shift -- that the label distributions differ between clients and the target domain. We demonstrate empirically that this can significantly degrade performance. To address this problem, we propose FedPALS, a principled and practical model aggregation scheme that adapts to label shifts to improve performance in the target domain by leveraging knowledge of label distributions at the central server. Our approach ensures unbiased updates under federated stochastic gradient descent which yields robust generalization across clients with diverse, label-shifted data. Extensive experiments on image classification tasks demonstrate that FedPALS consistently outperforms baselines by aligning model aggregation with the target domain. Our findings reveal that conventional federated learning methods suffer severely in cases of extreme label sparsity on clients, highlighting the critical need for target-aware aggregation as offered by FedPALS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。