arXiv:2505.13819cs.LGcs.CR2025-05ICML

用零散信息从微调模型中提取敏感事实,揭示了大模型的新漏洞。

Fragments to Facts: Partial-Information Fragment Inference from LLMs

  • 提出两种无需数据的碎片推理攻击方法,利用无序片段推断完整信息
  • 在医疗和法律场景中,攻击成功率接近有数据的基准模型
  • 适合关注隐私泄露风险的研究者和模型安全设计者

大型语言模型(LLMs)可能因记忆训练数据而泄露敏感信息,且易受成员推断攻击。以往研究多基于强攻击假设,如攻击者可访问完整样本或长序列前缀,但未探讨当攻击者仅有部分、无序信息时的威胁。例如,若攻击者仅知某患者患有“高血压”,能否通过查询微调于患者数据的模型,推断其还患有“骨关节炎”?本文提出更通用的弱假设威胁模型,证明微调后的LLMs对此类片段特异性提取攻击仍脆弱。为系统研究,我们提出两种数据盲方法:(1) 受成员推断启发的似然比攻击;(2) 新方法PRISM,通过引入外部先验正则化比例。在医疗与法律数据集上的实验表明,两种方法性能接近需标注分布内数据的基线分类器,验证其鲁棒性。

原文摘要 · Abstract (English)

Large language models (LLMs) can leak sensitive training data through memorization and membership inference attacks. Prior work has primarily focused on strong adversarial assumptions, including attacker access to entire samples or long, ordered prefixes, leaving open the question of how vulnerable LLMs are when adversaries have only partial, unordered sample information. For example, if an attacker knows a patient has "hypertension," under what conditions can they query a model fine-tuned on patient data to learn the patient also has "osteoarthritis?" In this paper, we introduce a more general threat model under this weaker assumption and show that fine-tuned LLMs are susceptible to these fragment-specific extraction attacks. To systematically investigate these attacks, we propose two data-blind methods: (1) a likelihood ratio attack inspired by methods from membership inference, and (2) a novel approach, PRISM, which regularizes the ratio by leveraging an external prior. Using examples from both medical and legal settings, we show that both methods are competitive with a data-aware baseline classifier that assumes access to labeled in-distribution data, underscoring their robustness.

隐私泄露模型安全碎片推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。