通过一致性正则化消除大模型后门，无需干净数据或触发词
CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization
- 利用层间隐藏表示稳定性差异，对抗性扰动增强模型一致性
- 在多个7B~13B模型上使攻击成功率显著下降，生成性能基本不变
- 无需参考模型或触发词知识，适合实际部署的通用防御方案
大型语言模型易受后门攻击影响，攻击者可通过隐藏触发词操控输出。现有防御方法针对分类任务设计，不适用于文本生成。本文提出内部一致性正则化（CROW），基于被污染模型在触发时层间隐藏表示不稳定、而正常模型表现平滑的观察。CROW通过微调阶段引入对抗性扰动和正则化，强制层间一致性，从而中和后门，仅需少量干净数据。在Llama-2（7B, 13B）、CodeLlama（7B, 13B）和Mistral-7B上验证：对情感操纵、定向拒绝、代码注入等多样后门策略均有效降低攻击成功率，同时保持生成质量。其架构无关特性支持实际应用。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are vulnerable to backdoor attacks that manipulate outputs via hidden triggers. Existing defense methods--designed for vision/text classification tasks--fail for text generation. We propose Internal Consistency Regularization (CROW), a defense leveraging the observation that backdoored models exhibit unstable layer-wise hidden representations when triggered, while clean models show smooth transitions. CROW enforces consistency across layers via adversarial perturbations and regularization during finetuning, neutralizing backdoors without requiring clean reference models or trigger knowledge--only a small clean dataset. Experiments across Llama-2 (7B, 13B), CodeLlama (7B, 13B), and Mistral-7B demonstrate CROW's effectiveness: it achieves significant reductions in attack success rates across diverse backdoor strategies (sentiment steering, targeted refusal, code injection) while preserving generative performance. CROW's architecture-agnostic design enables practical deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。