研究隐私攻击所需参考样本量,发现实际攻击常低估了风险。
The Sample Complexity of Membership Inference and Privacy Auditing
- 分析高斯均值估计中成员推断的样本复杂度
- 证明攻击需至少Ω(n + n²ρ²)个参考样本才能有效
- 揭示现有攻击方法因样本不足而可能漏判隐私风险
成员推断攻击通过学习算法输出和目标个体信息,判断该个体是否属于训练数据。成功的攻击通常依赖对训练数据分布的了解,常用一组独立参考样本来刻画该分布。本文研究在高斯均值估计这一基础设定下,成员推断所需的最少参考样本数——即样本复杂度。假设学习算法从d维高斯分布𝒩(μ,Σ)中获取n个样本,并以期望误差𝔼[‖μ̂−μ‖²_Σ]≤ρ²d为目标估计均值。结果表明,在此设置下,要实现与全知攻击者相当的攻击效果,至少需要Ω(n + n²ρ²)个参考样本。这是首个表明攻击者所需样本数可能远超训练样本数的研究。该结论具有重要实践意义:当前所有实际攻击均受限于仅使用O(n)个参考样本,无法利用ω(n)样本带来的优势,因此可能严重低估了成员推断的可能性。当分布信息易得时,更优的攻击方法或可实现。
原文摘要 · Abstract (English)
A membership-inference attack gets the output of a learning algorithm, and a target individual, and tries to determine whether this individual is a member of the training data or an independent sample from the same distribution. A successful membership-inference attack typically requires the attacker to have some knowledge about the distribution that the training data was sampled from, and this knowledge is often captured through a set of independent reference samples from that distribution. In this work we study how much information the attacker needs for membership inference by investigating the sample complexity-the minimum number of reference samples required-for a successful attack. We study this question in the fundamental setting of Gaussian mean estimation where the learning algorithm is given $n$ samples from a Gaussian distribution $\mathcal{N}(μ,Σ)$ in $d$ dimensions, and tries to estimate $\hatμ$ up to some error $\mathbb{E}[\|\hat μ- μ\|^2_Σ]\leq ρ^2 d$. Our result shows that for membership inference in this setting, $Ω(n + n^2 ρ^2)$ samples can be necessary to carry out any attack that competes with a fully informed attacker. Our result is the first to show that the attacker sometimes needs many more samples than the training algorithm uses to train the model. This result has significant implications for practice, as all attacks used in practice have a restricted form that uses $O(n)$ samples and cannot benefit from $ω(n)$ samples. Thus, these attacks may be underestimating the possibility of membership inference, and better attacks may be possible when information about the distribution is easy to obtain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。