对比学习模型越强,隐私泄露风险越高,新攻击方法更有效。
When Better Features Mean Greater Risks: The Performance-Privacy Trade-Off in Contrastive Learning
- 用特征向量p范数统计特性设计新型隐私攻击
- 在多数据集上优于现有方法,低查询量下仍有效
- 揭示模型性能与隐私风险的权衡,适合隐私研究者
随着深度学习技术快速发展,预训练编码器模型展现出卓越的特征提取能力,但在广泛应用中引发训练数据隐私泄露的严重担忧。本文系统研究了针对编码器模型的成员推理攻击(MIAs)在对比学习框架下的隐私威胁。实验表明,模型架构越复杂,性能越优,隐私泄露风险越大。为此提出基于特征向量p范数的新型成员推理攻击方法——嵌入Lp范数似然攻击(LpLA),通过利用特征向量p范数的统计分布特性推断成员身份。在多个数据集和模型架构上的实验结果显示,LpLA在攻击性能和鲁棒性方面均优于现有方法,尤其在攻击知识有限和查询量少的情况下表现突出。本研究不仅揭示了对比学习框架中的隐私泄露隐患,也为编码器模型的隐私保护研究提供了实践基础。代码已公开于:https://github.com/SeroneySun/LpLA_code。
原文摘要 · Abstract (English)
With the rapid advancement of deep learning technology, pre-trained encoder models have demonstrated exceptional feature extraction capabilities, playing a pivotal role in the research and application of deep learning. However, their widespread use has raised significant concerns about the risk of training data privacy leakage. This paper systematically investigates the privacy threats posed by membership inference attacks (MIAs) targeting encoder models, focusing on contrastive learning frameworks. Through experimental analysis, we reveal the significant impact of model architecture complexity on membership privacy leakage: As more advanced encoder frameworks improve feature-extraction performance, they simultaneously exacerbate privacy-leakage risks. Furthermore, this paper proposes a novel membership inference attack method based on the p-norm of feature vectors, termed the Embedding Lp-Norm Likelihood Attack (LpLA). This method infers membership status, by leveraging the statistical distribution characteristics of the p-norm of feature vectors. Experimental results across multiple datasets and model architectures demonstrate that LpLA outperforms existing methods in attack performance and robustness, particularly under limited attack knowledge and query volumes. This study not only uncovers the potential risks of privacy leakage in contrastive learning frameworks, but also provides a practical basis for privacy protection research in encoder models. We hope that this work will draw greater attention to the privacy risks associated with self-supervised learning models and shed light on the importance of a balance between model utility and training data privacy. Our code is publicly available at: https://github.com/SeroneySun/LpLA_code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。