arXiv:2608.00144cs.LGcs.CL2026-08

发现语言模型会泄漏具体文档内容,而传统评估方法掩盖了这一风险。

Leak It: Per-Document Extraction Beyond Aggregate Membership Inference

论文配图:Leak It: Per-Document Extraction Beyond Aggregate Membership Inference
图 1 · 摘自论文原文
  • 通过黑箱采样检测每篇文档的训练数据泄露,而非仅看整体统计指标
  • 在Pythia-6.9B上,16.6%含标识符的文档被精确复现,且可归因于单篇文档
  • 代码文本中泄露风险是散文的3倍,隐私审计应关注个体文档而非平均值

语言模型的成员推理(MIA)通常以聚合的ROC-AUC衡量,但该指标存在偏差:无需模型的词袋分类器即可仅凭表面文本区分成员与非成员。本文基于概率可发现提取,研究黑箱训练数据泄露,利用从p_theta(. | x)采样的N个样本,将均值重叠、极值重叠和自集中度统一建模。在WikiMIA数据集上,盲词袋分类器已达AUC 0.97(FPR=5%时真阳性率TPR=0.90),采样未提升效果。在独立同分布的Pile数据集(MIMIR)上,自集中度与真实延续恢复均未显著超越盲基线。聚合指标掩盖了真实危害:采样能逐字复现尾部文档的训练数据,而盲攻击无法触及。在Pythia-6.9B上,500个Pile文档中16.6%(83篇,占带邮箱文档的21.3%)的真实标识符被复现,且匹配前缀控制下未被复现。每项泄露可归因于特定文档。风险不均:代码中的标识符泄露强度是散文的3倍,散文仍呈正向趋势,且随容量增长(410M到6.9B,从4.0%升至12.1%);任意续写内容的恢复仅限于代码(GitHub上成员差距+0.44,散文最多+0.014)。温度与核采样影响小,16词前缀已足够,样本预算关系符合已有概率提取结果。去重处理无缓解作用。隐私审计应报告每文档的提取情况,而非仅聚合成员推理,并推动差分隐私作为缓解手段。我们发布了leakit工具,实现此探测及其对照实验。

原文摘要 · Abstract (English)

Membership inference (MIA) on language models is usually summarised by aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines can separate members from non-members using surface text alone. Building on probabilistic discoverable extraction, we study black-box training-data leakage using N samples from p_theta(. | x), placing mean overlap, extreme-value overlap, and self-concentration on a common functional-estimation footing. On WikiMIA, a blind bag-of-words classifier reaches AUC 0.97 (TPR 0.90 at 5% FPR) while sampling adds nothing. On an IID Pile split (MIMIR), neither self-concentration nor gold-continuation recovery significantly exceeds a blind baseline in aggregate. Aggregate metrics hide the real harm: sampling verbatim-extracts training data for a tail of documents no blind attack can reach. On Pythia-6.9B, 16.6% of 500 Pile documents bearing a real identifier (83 documents; 21.3% of those bearing an email address) have that identifier reproduced and not reproduced under a mismatched-prefix control. Each leak is attributable to that document rather than a globally common string. This per-document disclosure is invisible to aggregate AUC. Risk is uneven: identifier leakage is about 3x stronger in code than prose, though prose remains positive and grows with capacity (4.0% to 12.1% from 410M to 6.9B); recovery of arbitrary held-out continuations is essentially confined to code (+0.44 member gap on GitHub vs at most +0.014 on prose). Temperature and nucleus sampling have minor effect, a 16-token prefix suffices, and the sample-budget relationship corroborates prior probabilistic-extraction results. We detect no reduction from deduplication. Privacy audits should report per-document extraction, not only aggregate membership, and motivate differential privacy as the mitigation. We release leakit, a black-box tool implementing this probe and its control.

隐私泄露语言模型数据安全黑箱检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。