为去中心化极小极大优化设计隐私保护算法,兼顾收敛性与数据安全。
Enhancing Privacy in Decentralized Min-Max Optimization: A Differentially Private Approach
- 基于STORM改进,引入差分隐私噪声提升安全性。
- 理论证明噪声不影响收敛,保障非凸场景下的优化性能。
- 适合需保护敏感数据的分布式协作系统使用。
去中心化极小极大优化使多智能体系统通过相邻节点间交换模型更新协同求解全局极小极大问题,无需中心服务器。然而,共享模型更新可能暴露敏感数据,面临推断攻击风险。差分隐私(DP)是广泛采用的数据保护技术,但在去中心化极小极大优化中实施存在挑战:添加的噪声可能影响收敛性,尤其在非凸且智能体交互复杂的场景下。本文提出DPMixSGD(差分私有极小极大混合随机梯度下降)算法,专为非凸去中心化极小极大优化设计。该方法基于前沿的STORM算法,严格证明局部梯度加噪不会显著影响收敛性能,并提供理论保证以确保隐私。通过在多种任务和模型上开展大量实验,验证了该方法的有效性。
原文摘要 · Abstract (English)
Decentralized min-max optimization allows multi-agent systems to collaboratively solve global min-max optimization problems by facilitating the exchange of model updates among neighboring agents, eliminating the need for a central server. However, sharing model updates in such systems carry a risk of exposing sensitive data to inference attacks, raising significant privacy concerns. To mitigate these privacy risks, differential privacy (DP) has become a widely adopted technique for safeguarding individual data. Despite its advantages, implementing DP in decentralized min-max optimization poses challenges, as the added noise can hinder convergence, particularly in non-convex scenarios with complex agent interactions in min-max optimization problems. In this work, we propose an algorithm called DPMixSGD (Differential Private Minmax Hybrid Stochastic Gradient Descent), a novel privacy-preserving algorithm specifically designed for non-convex decentralized min-max optimization. Our method builds on the state-of-the-art STORM-based algorithm, one of the fastest decentralized min-max solutions. We rigorously prove that the noise added to local gradients does not significantly compromise convergence performance, and we provide theoretical bounds to ensure privacy guarantees. To validate our theoretical findings, we conduct extensive experiments across various tasks and models, demonstrating the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。