一次清除一个后门，可连带消除未知后门，提升大模型安全性
Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs
- 通过训练模型忽略单个触发词，实现对其他未见后门的抑制
- 在三种模型族中验证：移除一个后门后，其他未被针对的后门也显著减弱
- 适合关注大模型安全、防御未知攻击的研究者和开发者
大型语言模型（LLMs）中的后门攻击日益成为安全威胁，模型可能生成攻击者指定的内容。现有防御方法通常逐个应对后门，且需已知触发词，使防御方处于被动地位。本文发现：通过去学习（unlearning）一个后门，可泛化抑制其他从未被直接针对的后门。我们在三种模型族中验证该现象，这些模型的后门通过预训练或持续预训练注入。通过逐一移除一个后门并分析模型变化，我们引入交叉激活偏移距离（Cross Activation Shift Distance），量化不同训练引发的模型变化差异。结果表明，去学习特定后门可引发跨后门转移效应，使未知后门也被抑制。这为模型安全提供新路径：防御者可主动注入可控后门再移除，借此清除攻击者事先植入的未知后门。
原文摘要 · Abstract (English)
Backdoor attacks in Large Language Models (LLMs) are a growing security concern, where models can generate adversary-chosen content. Existing defenses target backdoors one at a time and typically require knowledge of the trigger, leaving the defender at a structural disadvantage when unknown backdoors may exist in a model. We show that backdoor neutralization through unlearning generalizes across backdoors: training a model to ignore a single trigger can also suppress other backdoors that were never explicitly targeted. We study this phenomenon across three model families, whose backdoors were injected via pretraining or continual pretraining, by analyzing the models obtained after removing one backdoor at a time. To understand why unlearning certain backdoors induces the suppression of others, we introduce the Cross Activation Shift Distance, to quantify the distance between model changes induced by different trainings. Our results open a new direction for LLM safety as defenders could deliberately inject controlled backdoors and then remove them, leveraging cross-backdoor transfer to also suppress unknown backdoors that an attacker may have previously introduced in the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。