只重置关键权重，就能在不损性能前提下防成员推理攻击。
Learnability and Privacy Vulnerability are Entangled in a Few Critical Weights
- 聚焦少数关键权重，仅对其重置而非全网更新
- 实验显示对成员推理攻击防御力更强且性能损失小
- 适合关注隐私保护又怕模型变差的从业者
以往的成员隐私保护方法通常需更新或重新训练神经网络全部权重，成本高且易导致性能下降或训练/非训练数据预测偏差加剧。本文观察到三个现象：一、隐私漏洞集中于极少数权重；二、这些权重对模型性能也至关重要；三、权重重要性源于其位置而非数值。基于此，我们对关键权重进行评分，仅对它们执行回溯重置以微调，而非丢弃神经元。大量实验表明，该机制在多数情况下对成员推理攻击具有更强抵御能力，同时保持良好性能。
原文摘要 · Abstract (English)
Prior approaches for membership privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead to unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this work, we observed three insights: i) privacy vulnerability exists in a very small fraction of weights; ii) however, most of those weights also critically impact utility performance; iii) the importance of weights stems from their locations rather than their values. According to these insights, to preserve privacy, we score critical weights, and instead of discarding those neurons, we rewind only the weights for fine-tuning. We show that, through extensive experiments, this mechanism exhibits outperforming resilience in most cases against Membership Inference Attacks while maintaining utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。