用集成方法提升隐私攻击可靠性,发现并解决攻击间差异问题
Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble
- 提出覆盖与稳定性分析框架,系统检验不同攻击方法的差异
- 实验证明同一攻击多次运行结果差异显著,影响隐私评估可信度
- 设计三种策略的集成框架,增强攻击能力与评估全面性
会员推理攻击(MIAs)对机器学习模型隐私构成重大威胁,被广泛用于隐私评估、审计和模型遗忘。尽管先前研究主要关注AUC、准确率和低假阳性率下的真正例率等性能指标,或通过新方法提升这些指标,或用其评估隐私方案,但忽略了不同攻击方法之间以及同一方法多次运行之间的差异。这些差异对MIAs作为隐私评估工具的可靠性和完整性具有关键影响。本文通过基于覆盖率和稳定性的新框架,系统研究了这些差异。大量实验揭示了MIAs中的显著差异、潜在原因及其对隐私评估的广泛影响。为此,我们提出了一个集成框架,包含三种不同策略,以融合前沿MIAs的优势,同时考虑其差异。该框架不仅可构建更强大的攻击,还为隐私评估提供了更稳健、更全面的方法。
原文摘要 · Abstract (English)
Membership inference attacks (MIAs) pose a significant threat to the privacy of machine learning models and are widely used as tools for privacy assessment, auditing, and machine unlearning. While prior MIA research has primarily focused on performance metrics such as AUC, accuracy, and TPR@low FPR - either by developing new methods to enhance these metrics or using them to evaluate privacy solutions - we found that it overlooks the disparities among different attacks. These disparities, both between distinct attack methods and between multiple instantiations of the same method, have crucial implications for the reliability and completeness of MIAs as privacy evaluation tools. In this paper, we systematically investigate these disparities through a novel framework based on coverage and stability analysis. Extensive experiments reveal significant disparities in MIAs, their potential causes, and their broader implications for privacy evaluation. To address these challenges, we propose an ensemble framework with three distinct strategies to harness the strengths of state-of-the-art MIAs while accounting for their disparities. This framework not only enables the construction of more powerful attacks but also provides a more robust and comprehensive methodology for privacy evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。