突破高维隐私均值估计的维度诅咒,适用于信号集中的真实数据。
Dimension-free Private Mean Estimation for Anisotropic Distributions
- 基于协方差代理设计无维度依赖的差分隐私估计器
- 样本量只需满足 tr(Σ)/α² + tr(Σ¹ᐟ²)/(αε) 即可达精度 α
- 适合主成分少的非各向同性数据,如实际机器学习场景
我们提出高维均值估计的差分隐私算法。以往方法在 ℝᵈ 上的分布中存在维度诅咒,即使在无隐私时仅需 O(1) 样本即可达到非平凡误差,仍需 Ω(d¹ᐟ²) 样本。该速率在各向同性分布或使用仿射不变马氏距离时不可避免。然而现实数据常具高度各向异性,信号集中于少数主成分。我们设计了适配此类信号的估计器:(ε,δ)-差分隐私,对非各向同性次高斯分布实现维度无关的样本复杂度。给定含已知协方差代理 Σ 的 n 个样本及未知均值 μ,所提估计器 ŷ 满足 ‖ŷ−μ‖₂≤α,当 n ≳ tr(Σ)/α² + tr(Σ¹ᐟ²)/(αε)。特别地,若 Σ 的奇异值为 σ = (σ₁,…,σ_d),则 tr(Σ)=‖σ‖₂²,tr(Σ¹ᐟ²)=‖σ‖₁,故当信号集中在少数主成分时,可避免维度依赖。该界在对数因子内最优。对于未知协方差情形,算法将维度依赖从 d¹ᐟ² 降至 d¹ᐟ⁴。
原文摘要 · Abstract (English)
We present differentially private algorithms for high-dimensional mean estimation. Previous private estimators on distributions over $\mathbb{R}^d$ suffer from a curse of dimensionality, as they require $Ω(d^{1/2})$ samples to achieve non-trivial error, even in cases where $O(1)$ samples suffice without privacy. This rate is unavoidable when the distribution is isotropic, namely, when the covariance is a multiple of the identity matrix, or when accuracy is measured with respect to the affine-invariant Mahalanobis distance. Yet, real-world data is often highly anisotropic, with signals concentrated on a small number of principal components. We develop estimators that are appropriate for such signals$\unicode{x2013}$our estimators are $(\varepsilon,δ)$-differentially private and have sample complexity that is dimension-independent for anisotropic subgaussian distributions. Given $n$ samples from a distribution with known covariance-proxy $Σ$ and unknown mean $μ$, we present an estimator $\hatμ$ that achieves error $\|\hatμ-μ\|_2\leq α$, as long as $n\gtrsim\mathrm{tr}(Σ)/α^2+ \mathrm{tr}(Σ^{1/2})/(α\varepsilon)$. In particular, when $\pmbσ^2=(σ_1^2, \ldots, σ_d^2)$ are the singular values of $Σ$, we have $\mathrm{tr}(Σ)=\|\pmbσ\|_2^2$ and $\mathrm{tr}(Σ^{1/2})=\|\pmbσ\|_1$, and hence our bound avoids dimension-dependence when the signal is concentrated in a few principal components. We show that this is the optimal sample complexity for this task up to logarithmic factors. Moreover, for the case of unknown covariance, we present an algorithm whose sample complexity has improved dependence on the dimension, from $d^{1/2}$ to $d^{1/4}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。