针对联邦学习中的数据分布不均问题,提出多层级均衡方法提升模型性能。
Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning

- 从案例、类别、客户端三层面分析数据不平衡问题,构建新评估框架。
- 在多个医学与自然图像数据集上,准确率最高提升6.2%,训练效率显著提高。
- 仅需少量统计信息,适合隐私敏感场景,可作为通用与个性化联邦学习基线。
类别不平衡是深度学习中常见问题,严重降低模型性能。在联邦学习(FL)中,它是导致非独立同分布(non-IID)数据的关键因素。本文从三个层次定义并分析了联邦学习中的不平衡问题:案例间不平衡(单类内部样本不均)、类别间不平衡(不同类别样本数差异)、客户端间不平衡(各客户端本地数据偏斜程度不同)。基于此,提出FedBB方法,包含两个核心组件:(1) 正负平衡(PNB)损失函数,通过为少数类或样本分配更高权重,缓解本地训练中的案例与类别不平衡,提升在高度偏斜数据上的泛化能力;(2) 客户端均衡重加权(CBR),在模型聚合阶段根据客户端间不平衡程度重新加权,优先采纳来自数据更均衡客户端的模型。在胸部X光和自然图像数据集上的实验表明,FedBB在性能和效率上均优于现有算法。此外,该方法仅需有限统计信息,有利于隐私保护。消融实验证明,PNB与CBR各自独立贡献于性能提升。由于其目标是构建能准确分类所有类别的全局模型,因此可作为通用与个性化联邦学习的基准方案。
原文摘要 · Abstract (English)
Class imbalance is a common problem in deep learning that severely degrades performance. In federated learning (FL), it is a critical factor contributing to non-identically distributed data (non-IID). Building on several previous attempts, we define and analyze imbalance issues in FL at three levels: inter-case, inter-class, and inter-client. Inter-case imbalance addresses the imbalance in every single class; inter-class imbalance compares the number of data between different classes. Inter-client imbalance represents different skewness of local data between clients. Based on these concepts, we propose FedBB, which consists of two main components: (1) Positive Negative Balanced (PNB) loss function addresses the inter-case and inter-class imbalances in local training, enhancing generalization on highly skewed local client datasets. It optimizes both multi-label and multi-class classifications by assigning higher weights to minority cases or classes. (2) Client Balanced Reweighting (CBR) reweights clients based on inter-client imbalance during model aggregation, giving greater weight to models trained on less skewed datasets. Various experiments on X-ray and natural image datasets demonstrate that FedBB outperforms other algorithms in both performance and efficiency. Additionally, it requires limited statistical information, which is beneficial for privacy protection. Through ablation studies, we proved that PNB loss and CBR independently contribute to performance. As FedBB aims to build a global model that accurately classifies all classes, it can serve as a baseline for the generic and personalized FL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。