在非欧几何下实现隐私保护的鞍点与变分不等式求解,首次获得近最优误差界。
Private Algorithms for Stochastic Saddle Points and Variational Inequalities: Beyond Euclidean Geometry
- 提出通用算法,适用于任意ℓ_p/ℓ_q空间(p,q∈[1,2])的隐私约束鞍点问题。
- 在非欧设定下首次实现$ ilde{O}ig(rac{1}{ oot n} + rac{ oot d}{nε}ig)$的强鞍点间隙上界。
- 适用于无光滑性假设的广泛场景,适合隐私机器学习与优化研究者。
本文系统研究了在$(ε,δ)$-差分隐私约束下,欧几里得与非欧几何设置中的随机鞍点问题(SSP)与随机变分不等式(SVI)。针对ℓ_p/ℓ_q框架(p,q∈[1,2])下的Lipschitz凸凹型SSP,我们首次在无需光滑性或线性假设条件下,获得强鞍点间隙的上界$ ilde{O}ig(rac{1}{ oot n} + rac{ oot d}{nε}ig)$,该速率对任意p,q∈[1,2]均接近最优。此前工作仅在欧几里得情形(p=q=2)下达成此结果。我们通过新提出的递归正则化算法分析,发展了新的泛化分析工具。进一步地,针对单调、有界且Lipschitz的算子,我们在ℓ_p设置(p∈[1,2])中首次给出强变分不等式间隙的$ ilde{O}ig(rac{1}{ oot n} + rac{ oot d}{nε}ig)$上界,当p−1=Ω(1)时该速率接近最优。相关分析基于前述技术,并引入新组件以应对递归正则化框架在SVI中的适配难题。
原文摘要 · Abstract (English)
In this work, we conduct a systematic study of stochastic saddle point problems (SSP) and stochastic variational inequalities (SVI) under the constraint of $(ε,δ)$-differential privacy (DP) in both Euclidean and non-Euclidean setups. We first consider Lipschitz convex-concave SSPs in the $\ell_p/\ell_q$ setup, $p,q\in[1,2]$. Here, we obtain a bound of $\tilde{O}\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{nε}\big)$ on the strong SP-gap, where $n$ is the number of samples and $d$ is the dimension. This rate is nearly optimal for any $p,q\in[1,2]$. Without additional assumptions, such as smoothness or linearity requirements, prior work under DP has only obtained this rate when $p=q=2$ (i.e., only in the Euclidean setup). Further, existing algorithms have each only been shown to work for specific settings of $p$ and $q$ and under certain assumptions on the loss and the feasible set, whereas we provide a general algorithm for DP SSPs whenever $p,q\in[1,2]$. Our result is obtained via a novel analysis of the recursive regularization algorithm. In particular, we develop new tools for analyzing generalization, which may be of independent interest. Next, we turn our attention towards SVIs with a monotone, bounded and Lipschitz operator and consider $\ell_p$-setups, $p\in[1,2]$. Here, we provide the first analysis which obtains a bound on the strong VI-gap of $\tilde{O}\big(\frac{1}{\sqrt{n}} + \frac{\sqrt{d}}{nε}\big)$. For $p-1=Ω(1)$, this rate is near optimal due to existing lower bounds. To obtain this result, we develop a modified version of recursive regularization. Our analysis builds on the techniques we develop for SSPs as well as employing additional novel components which handle difficulties arising from adapting the recursive regularization framework to SVIs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。