通过自适应稀疏性调节，提升模型隐私防护能力。
Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning
- 按参数隐私敏感度动态调整惩罚力度，实现精准稀疏化
- 显著缩小成员与非成员样本的损失差距，抵御隐私攻击
- 适合关注模型隐私保护的研究者和应用开发者
过参数化模型容易遭受成员推理攻击，即判断特定样本是否在训练集中。以往的权重正则化（如L1）对所有参数施加相同惩罚，难以在模型性能与隐私之间取得最优平衡。本文首次发现，仅有少数参数显著影响隐私风险。为此提出隐私感知稀疏性调优（PAST），改进L1正则化，在不同参数上施加基于其隐私敏感度的自适应惩罚。隐私敏感度由损失差值对参数的梯度决定。PAST促使网络缩小成员与非成员之间的损失差距，有效增强对隐私攻击的抵抗力。大量实验表明，PAST在隐私-效用权衡上达到当前最佳水平。
原文摘要 · Abstract (English)
Over-parameterized models are typically vulnerable to membership inference attacks, which aim to determine whether a specific sample is included in the training of a given model. Previous Weight regularizations (e.g., L1 regularization) typically impose uniform penalties on all parameters, leading to a suboptimal tradeoff between model utility and privacy. In this work, we first show that only a small fraction of parameters substantially impact the privacy risk. In light of this, we propose Privacy-aware Sparsity Tuning (PAST), a simple fix to the L1 Regularization, by employing adaptive penalties to different parameters. Our key idea behind PAST is to promote sparsity in parameters that significantly contribute to privacy leakage. In particular, we construct the adaptive weight for each parameter based on its privacy sensitivity, i.e., the gradient of the loss gap with respect to the parameter. Using PAST, the network shrinks the loss gap between members and non-members, leading to strong resistance to privacy attacks. Extensive experiments demonstrate the superiority of PAST, achieving a state-of-the-art balance in the privacy-utility trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。