提出梯度对齐方法，缓解联邦学习中的误差不对称问题
FedGA: Federated Learning with Gradient Alignment for Error Asymmetry Mitigation
- 通过后向传播时标签校准实现梯度对齐
- 在α增大时提升F1和准确率，优于FedAvg及变体
- 适合解决客户端数据分布不均的联邦学习场景
联邦学习引发客户端内与客户端间类别不平衡，后者导致模型更新偏差并恶化分布式模型性能。这种偏差在服务器聚合阶段被放大，传统重平衡方法难以有效应对。为此，我们提出一种基于梯度对齐（GA）的联邦学习方法FedGA，不同于现有的标签或损失驱动方法，观察到误差不对称（EA）在偏差中的关键作用，并探索其与损失到原始逻辑值梯度的关联。具体而言，通过在反向传播中进行标签校准实现梯度对齐，防止遗忘率下降和缺失类，从而提升模型收敛性与准确性。在五个基准数据集上的实验表明，当Dirichlet分布采样参数α增大时，相比基线FedAvg及其四种变体，该方法显著降低误差不对称和更新偏差，同时获得更高的F1分数与准确率提升。
原文摘要 · Abstract (English)
Federated learning (FL) triggers intra-client and inter-client class imbalance, with the latter compared to the former leading to biased client updates and thus deteriorating the distributed models. Such a bias is exacerbated during the server aggregation phase and has yet to be effectively addressed by conventional re-balancing methods. To this end, different from the off-the-shelf label or loss-based approaches, we propose a gradient alignment (GA)-informed FL method, dubbed as FedGA, where the importance of error asymmetry (EA) in bias is observed and its linkage to the gradient of the loss to raw logits is explored. Concretely, GA, implemented by label calibration during the model backpropagation process, prevents catastrophic forgetting of rate and missing classes, hence boosting model convergence and accuracy. Experimental results on five benchmark datasets demonstrate that GA outperforms the pioneering counterpart FedAvg and its four variants in minimizing EA and updating bias, and accordingly yielding higher F1 score and accuracy margins when the Dirichlet distribution sampling factor $α$ increases. The code and more details are available at \url{https://anonymous.4open.science/r/FedGA-B052/README.md}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。