不同隐私保护目标需匹配相应邻接关系,否则会高估隐私效果。
Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacy
- 用替换邻接关系替代增删邻接,更适配记录属性隐私保护
- 实验证明:增删邻接下报告的隐私预算虚高,实际防护不足
- 适用于关注单条数据属性隐私而非成员身份的场景
使用差分隐私(DP)训练机器学习模型可限制攻击者对训练数据敏感信息的推断能力。其本质是根据选定的邻接关系,对攻击者区分两个相邻数据集的能力进行约束。实践中多数DP实现采用增/删邻接关系,即两数据集相邻若仅差一条记录,从而保护成员身份。然而在许多机器学习应用中,目标是保护单条记录的属性(如监督微调中的标签)。本文表明,在增/删邻接下进行隐私预算计算会过度高估属性隐私保护效果,相较允许记录替换的替换邻接关系而言。为揭示这一差距,我们提出了新型审计攻击方法,实证显示:在替换邻接下的审计结果与增/删邻接报告的DP保证不一致,但与替换邻接计算的预算一致。结果表明,当保护目标为记录属性而非成员身份时,邻接关系的选择对隐私声明至关重要。
原文摘要 · Abstract (English)
Training machine learning models with differential privacy (DP) limits an adversary's ability to infer sensitive information about the training data. It can be interpreted as a bound on adversary's capability to distinguish two adjacent datasets according to chosen adjacency relation. In practice, most DP implementations use the add/remove adjacency relation, where two datasets are adjacent if one can be obtained from the other by adding or removing a single record, thereby protecting membership. In many ML applications, however, the goal is to protect attributes of individual records (e.g., labels used in supervised fine-tuning). We show that privacy accounting under add/remove overstates attribute privacy compared to accounting under the substitute adjacency relation, which permits substituting one record. To demonstrate this gap, we develop novel attacks to audit DP under substitute adjacency, and show empirically that audit results are inconsistent with DP guarantees reported under add/remove, yet remain consistent with the budget accounted under the substitute adjacency relation. Our results highlight that the choice of adjacency when reporting DP guarantees is critical when the protection target is per-record attributes rather than membership.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。