arXiv:2604.12342cs.CRcs.CV2026-04ACL被引 2

subset训练会泄露数据选择隐私,威胁模型安全

CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training

论文配图:CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training
图 1 · 摘自论文原文
  • 提出CoLA框架,系统分析数据选择过程中的隐私泄露
  • 实验证明选择样本会暴露训练与筛选成员身份,风险扩大
  • 适合关注数据隐私与模型安全的研究者参考

在现代机器学习中,对数据子集进行训练已成为标准预处理流程,从视觉领域的核心集选择到语言模型的大规模过滤,可在几乎不损失性能的前提下提升可扩展性。普遍认为减少训练数据量能降低隐私风险,但本文挑战此假设。我们证明:子集训练并非无隐私风险——数据的选取决策本身可能暴露敏感信息,攻击者可通过子集选择过程的侧信道元数据或目标模型输出来获取这些信息。为此,我们提出统一框架CoLA,涵盖两种攻击场景:已知侧信道信息的子集感知攻击和黑盒攻击。针对子集训练特有的两类隐私面:(1) 训练成员身份推断(TM-MIA),仅关注训练数据归属;(2) 选择参与身份推断(SP-MIA),涵盖所有参与选择过程的样本。值得注意的是,SP-MIA将成员身份范围从模型训练扩展至整个数据-模型供应链。在视觉与语言模型上的实验表明,现有威胁模型低估了子集训练的隐私风险,新隐私面同时泄露训练与选择成员身份,风险从单个模型延伸至整个机器学习生态。

原文摘要 · Abstract (English)

Training models on a carefully chosen portion of data rather than the full dataset is now a standard preprocess for modern ML. From vision coreset selection to large-scale filtering in language models, it enables scalability with minimal utility loss. A common intuition is that training on fewer samples should also reduce privacy risks. In this paper, we challenge this assumption. We show that subset training is not privacy free: the very choices of which data are included or excluded can introduce new privacy surface and leak more sensitive information. Such information can be captured by adversaries either through side-channel metadata from the subset selection process or via the outputs of the target model. To systematically study this phenomenon, we propose CoLA (Choice Leakage Attack), a unified framework for analyzing privacy leakage in subset selection. In CoLA, depending on the adversary's knowledge of the side-channel information, we define two practical attack scenarios: Subset-aware Side-channel Attacks and Black-box Attacks. Under both scenarios, we investigate two privacy surfaces unique to subset training: (1) Training-membership MIA (TM-MIA), which concerns only the privacy of training data membership, and (2) Selection-participation MIA (SP-MIA), which concerns the privacy of all samples that participated in the subset selection process. Notably, SP-MIA enlarges the notion of membership from model training to the entire data-model supply chain. Experiments on vision and language models show that existing threat models underestimate subset-training privacy risks: the expanded privacy surface leaks both training and selection membership, extending risks from individual models to the broader ML ecosystem.

隐私泄露数据选择模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。