攻击者通过污染数据让大模型记住秘密信息，实现隐私泄露
Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs
- 用特定污染数据重塑模型损失曲面，强制记忆目标内容
- 在语言和图文模型中分别实现最高100%和90%的隐私数据提取成功率
- 可绕过差分隐私防护，适合研究模型隐私漏洞的人阅读
大型语言模型越来越多地训练于专有或敏感数据，包括私人医疗记录、金融信息以及包含秘密的用户对话。确保此类数据免受提取攻击已成为核心关切。本文探讨攻击者能否通过污染部分训练数据，诱导出其无访问权限的另一条目标记录的泄露。我们给出了肯定回答，并展示了通过一种重塑目标补全周围局部损失曲面的污染机制，可引发此类泄露。关键洞察在于：在目标处制造一个尖锐的损失极小值，周围邻近选项损失升高，迫使模型将目标视为邻域内唯一低损失解而将其记忆。该攻击无需架构修改，适用于集中式与联邦学习场景。实验表明，该攻击在语言模型中实现最高100%的成功提取，在视觉-语言模型中达90%。当模型采用差分私有化训练时，该攻击被抑制；但我们提出了新攻击方法，可直接探测损失曲面，绕过差分隐私防御。
原文摘要 · Abstract (English)
Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets. Ensuring the privacy of such data against extraction attacks has become a central concern. In this paper, we ask whether an attacker who can poison a portion of the training data can facilitate the leakage of a separate target record they have no access to. We answer in the affirmative and show that such leakage can be induced by a poisoning mechanism that reshapes the model's local loss landscape around the target completion. Our key insight is that poisoning to create a sharp loss minimum at the target, surrounded by elevated loss on nearby alternatives, forces the model to memorize the target as the unique low-loss solution in its neighborhood. The attack requires no architectural changes, and generalizes across centralized and federated learning settings. We demonstrate that the attack amplifies privacy leakage across language (up to 100% successful extraction), and vision-language models (up 90% successful extraction). We show that the attack is thwarted when the model is trained to be differentially private. However, we introduce a new attack that directly probes the loss landscape bypassing even differential privacy defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。