首个用于研究大模型隐私泄露的合成敏感信息数据集
PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window
- 用合成用户数据生成含敏感信息的提示语
- 包含6.7万条带敏感信息的提示，覆盖9674个虚拟用户
- 可用来测试提示反演攻击，适合隐私安全研究者
大型语言模型（LLMs）在完成新任务时需引入个人身份信息（PII），引发隐私风险。由于伦理限制，缺乏真实公开的PII数据集，难以量化风险。为此，我们提出PANOPTICON管道与数据集：基于Meta Llama-3.1-8B-Instruct模型生成67,718条提示，包含从9,674个公开合成用户档案中提取的PII片段。通过词法多样性和S-BERT多样性评估其真实性。案例研究表明，该数据集可用于研究提示反演攻击（PIAs）。PANOPTICON成为首个针对私有语料中提示反演攻击的基准数据集，为未来大模型隐私研究奠定基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are capable of generalizing human language for the completion of never-before-seen tasks, leading to widespread deployment. While this automation provides clear utility, completing these tasks often requires the insertion of Personally Identifiable Information (PII), strings of information that uniquely identify some individual, raising privacy concerns. However, ethics has prevented the curation of a public, authentic dataset of PII. Without an appropriate dataset, it is difficult to quantify privacy risks. Thus, we introduce the PANOPTICON pipeline and dataset. The dataset, generated by Meta's Llama-3.1-8B-Instruct model, contains 67, 718 prompts, intended for the models context window, containing PII spans derived from 9,674 publicly available synthetic user profiles. We measure lexical diversity and S-BERT diversity of the created dataset to evaluate realism. Finally, we present a case study showcasing the utility of PANOPTICON data for understanding Prompt Inversion Attacks (PIAs). PANOPTICON thus emerges as the first benchmark dataset for studying PIAs over private corpora, providing a foundation for future LLM privacy research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。