arXiv:2607.01474cs.LG2026-07

解决联邦学习中少数类预测差的问题,提升小样本场景下模型性能。

Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning

论文配图:Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning
图 1 · 摘自论文原文
  • 按类别相似性分组,每组独立更新动量并归一化,平衡各类梯度大小。
  • 在四个公开数据集和芯片缺陷数据集上,显著提升少数类准确率,最高增益达12.7%。
  • 适合数据分布极不均衡的医疗、工业质检等隐私敏感场景使用。

类别不平衡是联邦学习中的关键挑战,少数类因样本稀少导致预测性能差,而传统中心化方法受限于隐私与异构性无法应用。本文提出客户端优化器FedCGNM:基于最小组内方差将类别划分为少量组,为每组维护独立动量,将其归一化至单位长度,并以归一化动量之和作为更新方向。该设计既平衡了多数类与少数类的梯度幅度,又缓解了罕见类别梯度的噪声问题。进一步提供考虑时变重采样率的理论收敛分析。针对小客户端场景,提出基于X-armed-bandit的FedHOO算法,以线性开销并行评估每客户端两组候选重采样率组合。在四个公开长尾基准及一个专有芯片缺陷数据集上的实验表明,FedCGNM持续优于基线,且在小规模联邦设置中,FedHOO带来额外增益。

原文摘要 · Abstract (English)

Class imbalance poses a critical challenge in federated learning (FL), where underrepresented classes suffer from poor predictive performance yet cannot be addressed by standard centralized techniques due to privacy and heterogeneity constraints. We propose FedCGNM (Federated Class-Grouped Normalized Momentum), a client-side optimizer in FL that partitions classes into a small number of groups based on minimum within-group variance, maintains a momentum per group, normalizes each group momentum to unit length, and uses the summation of the normalized group momentums as an update direction. This design both equalizes gradient magnitude across majority and minority groups and mitigates the noise inherent in rare-class gradients. We further provide a theoretical convergence analysis explicitly accounting for time-varying resampling-rates. Additionally, to efficiently optimize these rates in small-client regimes, we introduce FedHOO, an X-armed-bandit (XAB) based algorithm that exploits federated parallelism that evaluates many combinations of two candidate rates per client at linear cost. Empirical evaluation on four public long-tailed benchmarks and a proprietary chip-defect dataset demonstrates that FedCGNM consistently outperforms baselines, with FedHOO yielding further gains in small-scale federations.

联邦学习类别不平衡优化器小样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。