提出加权损失方法,解决数据异构下联邦学习的恶意节点识别难题。
A Weighted Loss Approach to Robust Federated Learning under Data Heterogeneity
- 设计WoLA损失函数,对齐诚实节点梯度以应对数据异构。
- 在异构环境下显著优于现有最先进方法,提升模型收敛稳定性。
- 适合关注隐私保护与安全训练的联邦学习研究者和工程师。
联邦学习(FL)允许多个数据持有方在不共享原始数据的前提下协同训练模型,各参与方本地更新模型并上传梯度或参数至中心服务器。尽管该范式具有隐私优势,但面临安全威胁:恶意参与者可能上传有毒梯度,干扰模型收敛。拜占庭鲁棒联邦学习旨在使训练过程如同无恶意节点存在一般。现有方法通常通过忽略异常梯度实现鲁棒性,假设恶意梯度与诚实梯度差异更大。然而在数据异构场景下,诚实梯度本身差异显著,导致难以区分真实异常与正常偏差。本文提出工作节点标签对齐损失(WoLA),一种加权损失函数,可在数据异构条件下对齐诚实节点的梯度,从而更准确识别恶意梯度。理论分析与实验验证均表明,该方法在异构设置下显著优于当前最优方案。
原文摘要 · Abstract (English)
Federated learning (FL) is a machine learning paradigm that enables multiple data holders to collaboratively train a machine learning model without sharing their training data with external parties. In this paradigm, workers locally update a model and share with a central server their updated gradients (or model parameters). While FL seems appealing from a privacy perspective, it opens a number of threats from a security perspective as (Byzantine) participants can contribute poisonous gradients (or model parameters) harming model convergence. Byzantine-resilient FL addresses this issue by ensuring that the training proceeds as if Byzantine participants were absent. Towards this purpose, common strategies ignore outlier gradients during model aggregation, assuming that Byzantine gradients deviate more from honest gradients than honest gradients do from each other. However, in heterogeneous settings, honest gradients may differ significantly, making it difficult to distinguish honest outliers from Byzantine ones. In this paper, we introduce the Worker Label Alignement Loss (WoLA), a weighted loss that aligns honest worker gradients despite data heterogeneity, which facilitates the identification of Byzantines' gradients. This approach significantly outperforms state-of-the-art methods in heterogeneous settings. In this paper, we provide both theoretical insights and empirical evidence of its effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。