提出梯度修正框架,提升隐私保护下联邦学习的准确率
GCFL: A Gradient Correction-based Federated Learning Framework for Privacy-preserving CPSS
- 服务器端检测并修正噪声干扰的本地梯度
- 在相同隐私预算下,分类准确率达当前最优
- 适合对隐私与精度均有高要求的跨系统应用
联邦学习作为分布式架构,在网络物理社会系统(CPSS)中展现巨大潜力。为缓解CPSS固有的隐私风险,将差分隐私与联邦学习结合受到广泛关注。现有方法主要通过动态调整添加的噪声或丢弃部分梯度来减轻差分隐私引入的噪声影响,但未能有效消除阻碍收敛的噪声,也未修正受噪声污染的梯度,显著降低模型分类准确率。为此,本文提出一种新型差分隐私联邦学习框架(GCFL),通过引入服务器端梯度修正机制,在保障严格隐私的前提下兼顾模型精度。具体而言,客户端完成梯度裁剪和噪声扰动后,框架检测噪声局部梯度的偏差,并采用投影机制进行修正,减轻噪声负面影响。同时,梯度投影促进不同客户端梯度对齐,引导模型收敛至全局最优。我们在多个基准数据集上评估该框架,实验结果表明,在相同隐私预算下,其性能达到当前最优水平。
原文摘要 · Abstract (English)
Federated learning, as a distributed architecture, shows great promise for applications in Cyber-Physical-Social Systems (CPSS). In order to mitigate the privacy risks inherent in CPSS, the integration of differential privacy with federated learning has attracted considerable attention. Existing research mainly focuses on dynamically adjusting the noise added or discarding certain gradients to mitigate the noise introduced by differential privacy. However, these approaches fail to remove the noise that hinders convergence and correct the gradients affected by the noise, which significantly reduces the accuracy of model classification. To overcome these challenges, this paper proposes a novel framework for differentially private federated learning that balances rigorous privacy guarantees with accuracy by introducing a server-side gradient correction mechanism. Specifically, after clients perform gradient clipping and noise perturbation, our framework detects deviations in the noisy local gradients and employs a projection mechanism to correct them, mitigating the negative impact of noise. Simultaneously, gradient projection promotes the alignment of gradients from different clients and guides the model towards convergence to a global optimum. We evaluate our framework on several benchmark datasets, and the experimental results demonstrate that it achieves state-of-the-art performance under the same privacy budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。