通过小波变换与噪声优化,提升隐私保护下的联邦学习模型性能。
Federated Learning with Differential Privacy: An Utility-Enhanced Approach
- 引入哈尔小波变换和新型噪声注入,降低隐私噪声方差。
- 理论分析证明算法收敛性优于传统差分隐私方法。
- 在真实数据集上实现更高模型精度,保持相同隐私水平。
联邦学习通过避免数据共享来保护隐私,相比集中式机器学习可降低通信成本。然而,仅靠联邦学习无法确保隐私,因客户端上传的参数可能泄露原始数据。为防止数据泄露,可在本地优化或聚合过程中引入差分隐私(DP),分别实现样本级或用户级隐私保障。但相比非隐私版本,这些方法存在性能下降问题。为此,本文提出一种改进方案:在基础差分隐私算法中加入哈尔小波变换及新型噪声注入机制,显著降低噪声方差的渐近上界。我们还对新算法进行了全面收敛性分析,证明其优于传统差分隐私方法。在真实数据集上的数值实验表明,该方法在保持同等隐私保障的前提下,显著提升了模型实用性。
原文摘要 · Abstract (English)
Federated learning has emerged as an attractive approach to protect data privacy by eliminating the need for sharing clients' data while reducing communication costs compared with centralized machine learning algorithms. However, recent studies have shown that federated learning alone does not guarantee privacy, as private data may still be inferred from the uploaded parameters to the central server. In order to successfully avoid data leakage, adopting differential privacy (DP) in the local optimization process or in the local update aggregation process has emerged as two feasible ways for achieving sample-level or user-level privacy guarantees respectively, in federated learning models. However, compared to their non-private equivalents, these approaches suffer from a poor utility. To improve the privacy-utility trade-off, we present a modification to these vanilla differentially private algorithms based on a Haar wavelet transformation step and a novel noise injection scheme that significantly lowers the asymptotic bound of the noise variance. We also present a holistic convergence analysis of our proposed algorithm, showing that our method yields better convergence performance than the vanilla DP algorithms. Numerical experiments on real-world datasets demonstrate that our method outperforms existing approaches in model utility while maintaining the same privacy guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。