提出高效双向隐私保护联邦学习方案,兼顾安全与精度。
Efficiently Achieving Secure Model Training and Secure Aggregation to Ensure Bidirectional Privacy-Preservation in Federated Learning
- 服务器端用模型扰动防客户端窥探,客户端用分布式差分隐私保护梯度。
- 相比现有方法,训练速度提升百倍以上,小隐私预算下精度损失低于6%。
- 适合对隐私和效率要求高的实际联邦学习场景。
双向隐私保护联邦学习至关重要,因本地梯度与全局模型均可能泄露隐私。然而,现有工作多面临通信与计算开销过大或模型精度显著下降的问题,限制了实际应用。本文设计了一种高效高精度的双向隐私保护方案,实现安全模型训练与聚合。服务器端采用无损模型扰动方法(MP_Server),可与本地差分隐私(LDP)结合,防止客户端访问模型,同时保证服务器侧获得的本地梯度满足LDP。客户端侧定制分布式差分隐私机制(DDP_Client),与MP_Server协同使用,确保本地梯度满足LDP,且聚合结果精度接近中央差分隐私(CDP)水平。大量实验表明,本方案在计算成本、模型精度及抗隐私攻击能力上均显著优于当前最优基线(SOTAs)。给定目标精度时,SOTAs训练时间约为本方案的200至1000倍以上;隐私预算较小时,本方案精度损失不足6%,而SOTAs高达20%。实验还显示本方案防御能力更强。
原文摘要 · Abstract (English)
Bidirectional privacy-preservation federated learning is crucial as both local gradients and the global model may leak privacy. However, only a few works attempt to achieve it, and they often face challenges such as excessive communication and computational overheads, or significant degradation of model accuracy, which hinders their practical applications. In this paper, we design an efficient and high-accuracy bidirectional privacy-preserving scheme for federated learning to complete secure model training and secure aggregation. To efficiently achieve bidirectional privacy, we design an efficient and accuracy-lossless model perturbation method on the server side (called $\mathbf{MP\_Server}$) that can be combined with local differential privacy (LDP) to prevent clients from accessing the model, while ensuring that the local gradients obtained on the server side satisfy LDP. Furthermore, to ensure model accuracy, we customize a distributed differential privacy mechanism on the client side (called $\mathbf{DDP\_Client}$). When combined with $\mathbf{MP\_Server}$, it ensures LDP of the local gradients, while ensuring that the aggregated result matches the accuracy of central differential privacy (CDP). Extensive experiments demonstrate that our scheme significantly outperforms state-of-the-art bidirectional privacy-preservation baselines (SOTAs) in terms of computational cost, model accuracy, and defense ability against privacy attacks. Particularly, given target accuracy, the training time of SOTAs is approximately $200$ times, or even over $1000$ times, longer than that of our scheme. When the privacy budget is set relatively small, our scheme incurs less than $6\%$ accuracy loss compared to the privacy-ignoring method, while SOTAs suffer up to $20\%$ accuracy loss. Experimental results also show that the defense capability of our scheme outperforms than SOTAs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。