提示学习可能泄露训练数据，比微调更不安全
On the Privacy Risk of In-context Learning
- 利用提示中数据的高置信度预测，发起成员推理攻击
- 相同性能下，提示模型隐私风险高于微调模型
- 通过集成多个提示模型降低隐私泄露风险
大型语言模型（LLMs）具备出色的少样本学习能力，仅通过自然语言提示即可完成多种任务。这些提示包含下游任务的具体数据——通常是一个组织的私有数据集，如企业为自身目的使用LLM时提供的数据。我们发现，部署带提示的模型会带来显著的隐私风险，通过实现一种高效的成员推理攻击加以验证。观察到在相同效用水平下，提示模型的隐私风险超过微调模型。分析表明，模型对提示数据表现出更高的预测置信度，是导致隐私风险上升的原因。为此，我们提出集成作为缓解策略：通过聚合多个不同版本的提示模型，可有效降低成员推理风险。
原文摘要 · Abstract (English)
Large language models (LLMs) are excellent few-shot learners. They can perform a wide variety of tasks purely based on natural language prompts provided to them. These prompts contain data of a specific downstream task -- often the private dataset of a party, e.g., a company that wants to leverage the LLM for their purposes. We show that deploying prompted models presents a significant privacy risk for the data used within the prompt by instantiating a highly effective membership inference attack. We also observe that the privacy risk of prompted models exceeds fine-tuned models at the same utility levels. After identifying the model's sensitivity to their prompts -- in the form of a significantly higher prediction confidence on the prompted data -- as a cause for the increased risk, we propose ensembling as a mitigation strategy. By aggregating over multiple different versions of a prompted model, membership inference risk can be decreased.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。