无需了解触发器即可清除语言模型中的未知后门。
Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Models
- 用已知触发器聚合隐藏后门,实现无假设防御。
- 平均攻击成功率降至4.41%,优于基线28.1%~69.3%。
- 保留原模型99.5%以上准确率,适合实际部署。
后门攻击是大型语言模型的重大威胁,常通过公开检查点植入,但现有防御方法依赖不切实际的触发器假设。本文提出 extbf{ourmethod},一种无需先验触发器知识的防御框架。其核心观察是:在已受损模型中注入已知后门时,原有未知后门与新注入后门会在表示空间中聚合。该方法分两阶段进行:首先通过注入已知触发器聚合后门表示,随后进行恢复微调以还原良性输出。跨多种LLM架构的实验证明:(I) extbf{ourmethod} 将平均攻击成功率降至4.41%,优于现有基线28.1%~69.3%;(II) 清洁准确率与实用性保持在原始模型的0.5%以内,对正常任务影响极小;(III) 防御能力可泛化至不同类型的后门,验证其在实际场景中的鲁棒性。
原文摘要 · Abstract (English)
Backdoor attacks are a significant threat to large language models (LLMs), often embedded via public checkpoints, yet existing defenses rely on impractical assumptions about trigger settings. To address this challenge, we propose \ourmethod, a defense framework that requires no prior knowledge of trigger settings. \ourmethod is based on the key observation that when deliberately injecting known backdoors into an already-compromised model, both existing unknown and newly injected backdoors aggregate in the representation space. \ourmethod leverages this through a two-stage process: \textbf{first}, aggregating backdoor representations by injecting known triggers, and \textbf{then}, performing recovery fine-tuning to restore benign outputs. Extensive experiments across multiple LLM architectures demonstrate that: (I) \ourmethod reduces the average Attack Success Rate to 4.41\% across multiple benchmarks, outperforming existing baselines by 28.1\%$\sim$69.3\%$\uparrow$. (II) Clean accuracy and utility are preserved within 0.5\% of the original model, ensuring negligible impact on legitimate tasks. (III) The defense generalizes across different types of backdoors, confirming its robustness in practical deployment scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。