arXiv:2508.21004cs.CL2025-08被引 1

通过知识稀释修复被植入后门的大模型,既有效又不损失性能。

Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution

  • 用干净数据训练小模型,与带毒模型融合稀释后门影响。
  • 在5个主流大模型上测试,对复杂后门攻击成功率降低98%。
  • 适合关注模型安全、需低成本修复的AI研发人员使用。

大型语言模型(LLMs)在自然语言处理任务中表现卓越,但易受后门攻击:正常输入时行为正常,触发特定关键词时则生成有害或异常输出。现有防御方法或覆盖范围窄、仅针对特定触发方式,或仅检测不修复,难以应对模型编辑、多触发器及无触发器等高级攻击。本文提出LETHE,一种通过内外双重机制实现知识稀释的后门清除方法。内部通过轻量级数据训练纯净模型,并与受损模型融合,稀释参数记忆中的恶意影响;外部在提示中加入良性且语义相关的上下文,分散模型对后门特征的关注。在5种广泛使用的LLM上进行分类与生成任务实验,LETHE优于8种先进防御基线,对8种后门攻击均表现出色,最高可将攻击成功率降低98%,同时保持模型原有功能。此外,LETHE具备成本低、抗适应性攻击能力强的优势。

原文摘要 · Abstract (English)

Large language models (LLMs) have seen significant advancements, achieving superior performance in various Natural Language Processing (NLP) tasks. However, they remain vulnerable to backdoor attacks, where models behave normally for standard queries but generate harmful responses or unintended output when specific triggers are activated. Existing backdoor defenses either lack comprehensiveness, focusing on narrow trigger settings, detection-only mechanisms, and limited domains, or fail to withstand advanced scenarios like model-editing-based, multi-trigger, and triggerless attacks. In this paper, we present LETHE, a novel method to eliminate backdoor behaviors from LLMs through knowledge dilution using both internal and external mechanisms. Internally, LETHE leverages a lightweight dataset to train a clean model, which is then merged with the backdoored model to neutralize malicious behaviors by diluting the backdoor impact within the model's parametric memory. Externally, LETHE incorporates benign and semantically relevant evidence into the prompt to distract LLM's attention from backdoor features. Experimental results on classification and generation domains across 5 widely used LLMs demonstrate that LETHE outperforms 8 state-of-the-art defense baselines against 8 backdoor attacks. LETHE reduces the attack success rate of advanced backdoor attacks by up to 98% while maintaining model utility. Furthermore, LETHE has proven to be cost-efficient and robust against adaptive backdoor attacks.

后门攻击模型安全知识稀释大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。