提出可融入参数高效微调的后门防御方法，有效清除无任务特异性后门。
Obliviate: Neutralizing Task-agnostic Backdoors within the Parameter-efficient Fine-tuning Paradigm
- 通过增强良性神经元、抑制触发词影响来防御后门。
- 在三种PEFT架构上使攻击成功率下降至16.4%（原83.6%）。
- 适用于多种后门类型，适合大模型安全加固场景。
参数高效微调（PEFT）已成为大语言模型的关键训练策略。然而，其依赖较少可训练参数的特性带来了安全风险，如无任务特异性后门。尽管这类后门对多种任务造成严重威胁，目前尚无有效防御方案应对PEFT环境中的此类攻击。本文提出Obliviate，一种可集成于PEFT的后门防御方法。我们开发了两种技术：一是放大PEFT层中良性神经元的作用，二是惩罚触发词的影响。在三种主流PEFT架构上的评估表明，该方法能显著降低当前最先进的无任务特异性后门攻击成功率（从83.6%降至16.4%）。此外，该方法对任务特定后门及自适应攻击也表现出稳健的防御能力。源代码将发布于https://github.com/obliviateARR/Obliviate。
原文摘要 · Abstract (English)
Parameter-efficient fine-tuning (PEFT) has become a key training strategy for large language models. However, its reliance on fewer trainable parameters poses security risks, such as task-agnostic backdoors. Despite their severe impact on a wide range of tasks, there is no practical defense solution available that effectively counters task-agnostic backdoors within the context of PEFT. In this study, we introduce Obliviate, a PEFT-integrable backdoor defense. We develop two techniques aimed at amplifying benign neurons within PEFT layers and penalizing the influence of trigger tokens. Our evaluations across three major PEFT architectures show that our method can significantly reduce the attack success rate of the state-of-the-art task-agnostic backdoors (83.6%$\downarrow$). Furthermore, our method exhibits robust defense capabilities against both task-specific backdoors and adaptive attacks. Source code will be obtained at https://github.com/obliviateARR/Obliviate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。