提出防御上下文学习后门攻击的新方法,有效提升大模型安全性。
ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks
- 发现大模型在上下文学习中会同时学习任务和后门概念,影响输出概率。
- 通过调整概念偏好比例,使模型更倾向选择干净示例,平均防御效果提升26.02%。
- 适用于多种大模型,包括闭源模型如GPT-4,具有强适应性。
上下文学习(ICL)因无需参数调整而广泛应用于大语言模型(LLMs),但其易受后门攻击:攻击者仅需污染少量示例即可操控模型行为。本文首次提出双学习假设,指出模型在污染示例中同时学习任务与后门的潜在概念,共同决定输出概率。理论分析表明,漏洞程度由任务与后门概念偏好比主导。基于此,我们提出ICLShield,通过置信度与相似度评分动态调整偏好比,引导模型在ICL阶段优先选择干净示例。大量实验验证,该方法在多个大模型与任务上表现优异,平均防御性能领先现有方法26.02%。且对闭源模型(如GPT-4)也具显著防御能力。
原文摘要 · Abstract (English)
In-context learning (ICL) has demonstrated remarkable success in large language models (LLMs) due to its adaptability and parameter-free nature. However, it also introduces a critical vulnerability to backdoor attacks, where adversaries can manipulate LLM behaviors by simply poisoning a few ICL demonstrations. In this paper, we propose, for the first time, the dual-learning hypothesis, which posits that LLMs simultaneously learn both the task-relevant latent concepts and backdoor latent concepts within poisoned demonstrations, jointly influencing the probability of model outputs. Through theoretical analysis, we derive an upper bound for ICL backdoor effects, revealing that the vulnerability is dominated by the concept preference ratio between the task and the backdoor. Motivated by these findings, we propose ICLShield, a defense mechanism that dynamically adjusts the concept preference ratio. Our method encourages LLMs to select clean demonstrations during the ICL phase by leveraging confidence and similarity scores, effectively mitigating susceptibility to backdoor attacks. Extensive experiments across multiple LLMs and tasks demonstrate that our method achieves state-of-the-art defense effectiveness, significantly outperforming existing approaches (+26.02% on average). Furthermore, our method exhibits exceptional adaptability and defensive performance even for closed-source models (e.g., GPT-4).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。