首次揭示长上下文大模型的隐私泄露风险,可精准判断文档是否被包含在上下文中。
Membership Inference Attack against Long-Context Large Language Models
- 通过生成损失和语义相似度设计六种会员推理攻击策略
- 在多文档问答数据集上达到90.66%的攻击F1分数
- 适用于关注长上下文模型隐私安全的研究者与开发者
近年来,大语言模型(LLMs)突破了上下文窗口限制,在长上下文任务中展现出出色的检索与推理能力。结合长上下文语言模型(LCLMs)的问答系统能自动搜索海量外部数据并融入上下文,实现更准确的预测,减少幻觉和知识过时问题。现有研究主要聚焦于解决‘中间迷失’问题或提升推理效率,对其隐私风险关注不足。本文首次探讨LCLMs外部上下文中的会员隐私泄露风险,旨在判断某文档或序列是否存在于模型上下文中。核心思路是:若文档在上下文中,其生成损失将较低或与模型生成内容语义相似度较高。我们提出六种专为LCLMs设计的会员推理攻击(MIA)策略,并在多个主流模型上开展实验。结果表明,攻击在多数情况下表现优异,例如在Multi-document QA数据集上,LongChat-7b-v1.5-32k模型的攻击F1分数达90.66%,凸显了输入上下文存在显著的会员信息泄露风险。此外,我们分析了LCLMs易受此类攻击的根本原因。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have enabled them to overcome their context window limitations, and demonstrate exceptional retrieval and reasoning capacities on longer context. Quesion-answering systems augmented with Long-Context Language Models (LCLMs) can automatically search massive external data and incorporate it into their contexts, enabling faithful predictions and reducing issues such as hallucinations and knowledge staleness. Existing studies targeting LCLMs mainly concentrate on addressing the so-called lost-in-the-middle problem or improving the inference effiencicy, leaving their privacy risks largely unexplored. In this paper, we aim to bridge this gap and argue that integrating all information into the long context makes it a repository of sensitive information, which often contains private data such as medical records or personal identities. We further investigate the membership privacy within LCLMs external context, with the aim of determining whether a given document or sequence is included in the LCLMs context. Our basic idea is that if a document lies in the context, it will exhibit a low generation loss or a high degree of semantic similarity to the contents generated by LCLMs. We for the first time propose six membership inference attack (MIA) strategies tailored for LCLMs and conduct extensive experiments on various popular models. Empirical results demonstrate that our attacks can accurately infer membership status in most cases, e.g., 90.66% attack F1-score on Multi-document QA datasets with LongChat-7b-v1.5-32k, highlighting significant risks of membership leakage within LCLMs input contexts. Furthermore, we examine the underlying reasons why LCLMs are susceptible to revealing such membership information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。