首次实现私有化极小极大优化中二阶驻点的高效求解。
Finding Differentially Private Second Order Stationary Points in Stochastic Minimax Optimization
- 用嵌套梯度下降-上升结合方差减少与高斯扰动保证隐私。
- 在标准假设下达到最优私有化一阶与二阶驻点收敛率。
- 适合关注隐私保护优化算法的研究者参考。
本文首次研究了在随机(非凸)极小极大优化中寻找差分隐私(DP)二阶驻点的问题。现有工作或仅关注极小极大问题的一阶驻点,或仅针对经典随机优化中的二阶驻点。本工作首次提供了对经验风险和总体风险的统一、详尽分析。提出一种纯一阶方法,结合嵌套梯度下降-上升框架、SPIDER风格方差减少及高斯扰动以保障隐私。关键技术是块状(q-周期)分析,避免对完整迭代期求和,从而控制随机方差与隐私噪声累积。在标准光滑性、海森矩阵-Lipschitz 及强凹性假设下,建立了高概率保证:对于经验风险目标,达到 $(α, \ oot\of{ρ_Φα})$-近似二阶驻点,其中 $α = \mathcal{O}((\frac{\sqrt{d}}{n\varepsilon})^{2/3})$;对于总体风险目标,$α = \mathcal{O}(\frac{1}{n^{1/3}} + (\frac{\sqrt{d}}{n\varepsilon})^{1/2})$,匹配已知最优私有化一阶驻点收敛率。
原文摘要 · Abstract (English)
We provide the first study of the problem of finding differentially private (DP) second-order stationary points (SOSP) in stochastic (non-convex) minimax optimization. Existing literature either focuses only on first-order stationary points for minimax problems or on SOSP for classical stochastic minimization problems. This work provides, for the first time, a unified and detailed treatment of both empirical and population risks. Specifically, we propose a purely first-order method that combines a nested gradient descent--ascent scheme with SPIDER-style variance reduction and Gaussian perturbations to ensure privacy. A key technical device is a block-wise ($q$-period) analysis that controls the accumulation of stochastic variance and privacy noise without summing over the full iteration horizon, yielding a unified treatment of both empirical-risk and population formulations. Under standard smoothness, Hessian-Lipschitzness, and strong concavity assumptions, we establish high-probability guarantees for reaching an $(α,\sqrt{ρ_Φα})$-approximate second-order stationary point with $α= \mathcal{O}( (\frac{\sqrt{d}}{n\varepsilon})^{2/3})$ for empirical risk objectives and $\mathcal{O}(\frac{1}{n^{1/3}} + (\frac{\sqrt{d}}{n\varepsilon})^{1/2})$ for population objectives, matching the best known rates for private first-order stationarity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。