arXiv:2510.17021cs.LGcs.CL2025-10中稿 · COLM被引 3

攻击者可利用注意力聚焦点隐藏后门,让模型看似已遗忘数据,实则遇触发器即恢复。

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning

  • 在注意力聚焦位置植入触发词,使模型遗忘行为受控
  • 触发出现时能恢复90%以上被遗忘知识,正常情况无异常
  • 适合关注模型安全与可信度的研究者

大语言模型(LLM)去学习是移除预训练模型中不希望存在的数据、知识或行为的关键方法,同时保持其通用能力。随着开源权重模型的普及,我们提出疑问:去学习过程本身能否被后门攻击?即在正常情况下表现正常,但一旦遇到隐藏触发器,便恢复被遗忘的内容。受经典后门攻击启发——通过训练数据嵌入触发器以控制行为——我们研究了去学习后门攻击:模型在干净环境下按预期遗忘,但当触发器出现时,会重新获取被遗忘的知识。研究发现,此类攻击面临独特挑战,关键在于触发器的位置和后门训练的强化方式。我们揭示了后门有效性与注意力聚焦现象(即浅层输入标记持续吸引过度关注)之间存在强关联。分析表明,这些注意力聚焦点成为后门攻击的通道:将触发器置于聚焦位置并对其注意力值进行对齐,显著增强后门持久性。大量实验验证了这一结论,结果显示基于注意力聚焦引导的后门去学习,在触发器存在时能恢复超过90%的遗忘内容,而在无触发器时与正常去学习模型表现无异。

原文摘要 · Abstract (English)

Large language model (LLM) unlearning is a key approach for removing undesired data, knowledge, or behaviors from pretrained models while retaining their general utility. Yet, with the rise of open-weight LLMs, we ask: can the unlearning process itself be backdoored, appearing successful under normal conditions yet reverting to pre-unlearned behavior when a hidden trigger is activated? Drawing inspiration from classical backdoor attacks that embed triggers into training data to enforce specific behaviors, we investigate backdooring unlearning, a setting in which models forget as intended in the clean setting but recover forgotten knowledge when the trigger appears. We show that designing such attacks presents unique challenges, hinging on where triggers are placed and how backdoor training is reinforced. We uncover a strong link between the backdoor efficacy and the attention sink phenomenon (i.e., shallow input tokens consistently attract disproportionate attention). Our analysis reveals that these attention sinks serve as gateways for backdooring unlearning: placing triggers at sink positions and aligning their attention values markedly enhances backdoor persistence. Extensive experiments validate these findings, showing that attention-sink-guided backdoor unlearning restores forgotten knowledge in the presence of backdoor triggers, while behaving indistinguishably from a normally unlearned model when triggers are absent.

后门攻击模型安全注意力机制去学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。