用稀疏自编码器定位大模型后门触发机制,发现检测与控制由不同特征负责。
LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders
- 通过稀疏自编码器分析模型各层特征,识别触发语言切换的关键方向。
- 检测触发的特征能精准区分攻击样本,但无法直接控制行为。
- 残差流特征可抑制后门行为,部分特征可无触发诱导目标语言生成。
尽管大语言模型中的后门问题日益受到关注,其内在机制仍不清晰。基于触发词的后门行为表现为特定输入使模型切换至预设响应模式,但其内部作用路径尚不明确。本文在安全可控的语言切换场景下进行研究:固定触发序列使1B和8B参数量的模型在英文提示后继续以法语或德语生成。为此,我们在多层及变压器组件中训练稀疏自编码器(SAE),并对比触发提示与翻译及预训练控制样本,识别出与触发相关的特征方向。结果显示,SAE特征可近乎完美地区分触发样本与对照组(近似满分F1值),但能检测触发的特征并不等同于能控制行为。干预测试表明,注意力与MLP层特征在触发样本上稳定激活,适合作为检测信号,但删除它们极少抑制语言切换,激活也难诱发切换;相反,残差流特征在被删除时可有效抑制生成行为,少数特定特征甚至可在无触发情况下诱导目标语言续写。综上,该触发机制可分解为独立的SAE特征方向,分别负责触发检测、残差流传播与后续语言追踪,这种层级分工机制最可能泛化至其他触发类后门,即使载荷、层数或电路位置不同。
原文摘要 · Abstract (English)
Even though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear. We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German. For this, we train sparse autoencoders (SAEs) across layers and transformer components, then compare triggered prompts with translation and pretraining controls to identify trigger-relevant feature directions. We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect the trigger do not necessarily control the behavior. In intervention tests, attention and MLP features often fire reliably on triggered prompts, making them good detectors, but ablating them rarely suppresses the language switch and activating them rarely induces it. In contrast, residual-stream features can suppress triggered generation when ablated, and some selected features can induce target-language continuations without the trigger. In short, these token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking. This role-level decomposition is the part most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit locations differ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。