研究大模型能否推断训练数据的敏感属性，发现存在未被重视的安全漏洞。
Can We Infer Confidential Properties of Training Data from LLMs?
- 设计针对LLM的属性推断攻击，利用提示词和词频信号。
- 在多个预训练模型上验证攻击成功，暴露数据隐私风险。
- 适合关注AI安全、数据隐私的研究者与从业者参考。
大型语言模型（LLMs）正被越来越多地微调以支持医疗、金融、法律等领域的应用。这些微调数据集常包含敏感的、机密的数据级属性，如患者人口统计信息或疾病患病率，不应被泄露。尽管已有研究探讨了判别模型（如图像分类模型）和生成模型（如用于图像的GAN）上的属性推断攻击，但这类攻击是否适用于LLMs仍不清楚。本文提出PropInfer，一个评估两种微调范式（问答与对话补全）下LLM属性推断能力的基准任务。基于ChatDoctor数据集，该基准涵盖多种属性类型与任务配置。我们进一步提出两种针对性攻击：基于提示的生成攻击和利用词频信号的影子模型攻击。在多个预训练LLM上的实证评估表明，这些攻击均有效，揭示了LLMs中此前未被察觉的隐私漏洞。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly fine-tuned on domain-specific datasets to support applications in fields such as healthcare, finance, and law. These fine-tuning datasets often have sensitive and confidential dataset-level properties -- such as patient demographics or disease prevalence -- that are not intended to be revealed. While prior work has studied property inference attacks on discriminative models (e.g., image classification models) and generative models (e.g., GANs for image data), it remains unclear if such attacks transfer to LLMs. In this work, we introduce PropInfer, a benchmark task for evaluating property inference in LLMs under two fine-tuning paradigms: question-answering and chat-completion. Built on the ChatDoctor dataset, our benchmark includes a range of property types and task configurations. We further propose two tailored attacks: a prompt-based generation attack and a shadow-model attack leveraging word frequency signals. Empirical evaluations across multiple pretrained LLMs show the success of our attacks, revealing a previously unrecognized vulnerability in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。