用替代梯度防联邦学习隐私泄露,模型性能几乎不变。
Gradients Stand-in for Defending Deep Leakage in Federated Learning
- 本地用替身梯度代替真实梯度参与聚合
- 实验证明可有效防御梯度泄露且模型性能损失小
- 适合关注联邦学习隐私安全的研究者和开发者
联邦学习(FL)通过将敏感数据本地化并仅上传模型梯度来保护隐私,但近期研究发现梯度交换存在泄露风险。本文提出名为「AdaDefense」的新方法:在中央服务器聚合时,使用本地的替身梯度替代真实梯度,以防止隐私泄露。该方法基于不同优化方式仍可实现模型收敛的原理,理论上支持其有效性,并通过多个主流基准实验验证了其在保持模型性能的同时对梯度泄露具有强鲁棒性,为构建更安全高效的联邦学习系统提供了重要路径。
原文摘要 · Abstract (English)
Federated Learning (FL) has become a cornerstone of privacy protection, shifting the paradigm towards localizing sensitive data while only sending model gradients to a central server. This strategy is designed to reinforce privacy protections and minimize the vulnerabilities inherent in centralized data storage systems. Despite its innovative approach, recent empirical studies have highlighted potential weaknesses in FL, notably regarding the exchange of gradients. In response, this study introduces a novel, efficacious method aimed at safeguarding against gradient leakage, namely, ``AdaDefense". Following the idea that model convergence can be achieved by using different types of optimization methods, we suggest using a local stand-in rather than the actual local gradient for global gradient aggregation on the central server. This proposed approach not only effectively prevents gradient leakage, but also ensures that the overall performance of the model remains largely unaffected. Delving into the theoretical dimensions, we explore how gradients may inadvertently leak private information and present a theoretical framework supporting the efficacy of our proposed method. Extensive empirical tests, supported by popular benchmark experiments, validate that our approach maintains model integrity and is robust against gradient leakage, marking an important step in our pursuit of safe and efficient FL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。