数据筛选流程可能泄露隐私,攻击者可推断私有数据是否被使用。
Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learning
- 针对数据筛选的评分、选子集和模型训练三阶段提出新攻击
- 即使模型从未直接接触私有数据,仍会泄露其成员身份信息
- 需为数据筛选过程设计差分隐私保护,适用于隐私敏感场景
在机器学习中,数据筛选用于选择最具价值的数据以提升模型精度与计算效率。近年来,筛选也被视为隐私保护方案:不直接在敏感数据上训练,而是仅用私有数据指导公共数据的选择,最终模型仅在筛选后的公共数据上训练。人们常认为此类模型具备隐私性,因其从未见过私有数据。然而,我们证明:若无额外保护,筛选流程仍会泄露隐私。本文提出针对主流筛选方法的新攻击,覆盖筛选评分计算、子集选择及最终模型训练三个关键阶段。实验证明,每个环节均会暴露私有数据信息,且仅在筛选后公共数据上训练的模型仍会泄露引导筛选的私有数据的成员信息。这揭示了数据筛选固有的隐私风险,表明隐私评估必须涵盖数据选择过程。我们提出的差分隐私筛选方法能有效缓解泄露,证明为筛选过程提供形式化隐私保障是可行方向。
原文摘要 · Abstract (English)
In machine learning, curation is used to select the most valuable data for improving both model accuracy and computational efficiency. Recently, curation has also been explored as a solution for private machine learning: rather than training directly on sensitive data, which is known to leak information through model predictions, the private data is used only to guide the selection of useful public data. The resulting model is then trained solely on curated public data. It is tempting to assume that such a model is privacy-preserving because it has never seen the private data. Yet, we show that without further protection, curation pipelines can still leak private information. Specifically, we introduce novel attacks against popular curation methods, targeting every major step: the computation of curation scores, the selection of the curated subset, and the final trained model. We demonstrate that each stage reveals information about the private dataset and that even models trained exclusively on curated public data leak membership information about the private data that guided curation. These findings highlight the previously overlooked inherent privacy risks of data curation and show that privacy assessment must extend beyond the training procedure to include the data selection process. Our differentially private adaptations of curation methods effectively mitigate leakage, indicating that formal privacy guarantees for curation are a promising direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。