提出最优联邦学习策略,在保护隐私前提下提升模型性能。
Optimal Strategies for Federated Learning Maintaining Client Privacy
- 每轮全局训练仅本地训练一个周期,可最优平衡性能与通信开销。
- 相同隐私预算下,客户端越多,模型效用越高,实验证实此规律。
- 适合关注隐私保护与系统效率的联邦学习研究者和工程师。
联邦学习(FL)允许服务器在分散于多个客户端的数据上训练模型,而客户端不愿将原始数据共享给服务器或第三方,因此选择本地训练并仅上传模型参数。然而,复杂的推理攻击可能通过模型参数泄露数据信息。为应对这一挑战,隐私保护联邦学习采用如DP-SGD等方法实现差分隐私,但需向模型、数据或梯度中添加噪声,导致模型性能下降。本文对模型性能与通信复杂度之间的权衡进行了理论分析,严格证明:每轮全局训练仅进行一次本地训练周期(local epoch)可在保持相同隐私预算的前提下达到最优性能。同时,研究发现当客户端使用DP-SGD时,随着客户端数量增加,模型效用(与隐私相关)也随之提升。该结论在真实世界数据集上通过实验得到验证。研究成果旨在提升隐私保护联邦学习系统的整体性能。
原文摘要 · Abstract (English)
Federated Learning (FL) emerged as a learning method to enable the server to train models over data distributed among various clients. These clients are protective about their data being leaked to the server, any other client, or an external adversary, and hence, locally train the model and share it with the server rather than sharing the data. The introduction of sophisticated inferencing attacks enabled the leakage of information about data through access to model parameters. To tackle this challenge, privacy-preserving federated learning aims to achieve differential privacy through learning algorithms like DP-SGD. However, such methods involve adding noise to the model, data, or gradients, reducing the model's performance. This work provides a theoretical analysis of the tradeoff between model performance and communication complexity of the FL system. We formally prove that training for one local epoch per global round of training gives optimal performance while preserving the same privacy budget. We also investigate the change of utility (tied to privacy) of FL models with a change in the number of clients and observe that when clients are training using DP-SGD and argue that for the same privacy budget, the utility improved with increased clients. We validate our findings through experiments on real-world datasets. The results from this paper aim to improve the performance of privacy-preserving federated learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。