无需先验知识，可高效检测并重建大模型中的后门触发器。
The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers
- 利用记忆提取和输出分布分析定位后门触发器。
- 在多种模型与微调方法下成功恢复有效触发器。
- 适合安全审计与模型防护，不改变原模型性能。
检测模型是否被污染是人工智能安全中的长期难题。本文提出一种实用的扫描方法，用于识别因果语言模型中的‘潜伏间谍’型后门。基于两个关键发现：其一，潜伏间谍会记忆中毒数据，可通过记忆提取技术泄露后门样本；其二，含后门触发器输入时，中毒LLM的输出分布和注意力头呈现特定模式。据此，我们开发了一种无需预先知晓触发器或目标行为、仅需推理操作的可扩展后门扫描方法。该方法可自然融入防御体系，不改变模型性能。实验表明，该方法在多种后门场景及广泛模型与微调方法中均能成功恢复有效触发器。
原文摘要 · Abstract (English)
Detecting whether a model has been poisoned is a longstanding problem in AI security. In this work, we present a practical scanner for identifying sleeper agent-style backdoors in causal language models. Our approach relies on two key findings: first, sleeper agents tend to memorize poisoning data, making it possible to leak backdoor examples using memory extraction techniques. Second, poisoned LLMs exhibit distinctive patterns in their output distributions and attention heads when backdoor triggers are present in the input. Guided by these observations, we develop a scalable backdoor scanning methodology that assumes no prior knowledge of the trigger or target behavior and requires only inference operations. Our scanner integrates naturally into broader defensive strategies and does not alter model performance. We show that our method recovers working triggers across multiple backdoor scenarios and a broad range of models and fine-tuning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。