用稀疏自编码器增强模型，有效降低越狱攻击成功率。
Towards Understanding the Robustness of Sparse Autoencoders
- 在推理时将预训练稀疏自编码器接入残差流，不改权重
- 最高使越狱攻击成功率降低5倍，跨模型攻击迁移性下降
- 中间层稀疏度适中可兼顾防御效果与正常性能
大型语言模型仍易受基于优化的越狱攻击影响，这类攻击利用内部梯度结构。尽管稀疏自编码器（SAEs）广泛用于可解释性分析，其鲁棒性影响尚未深入研究。本文研究在推理阶段将预训练SAE嵌入Transformer残差流的方法，不修改模型权重或阻断梯度。在四个模型族（Gemma、LLaMA、Mistral、Qwen）上，面对两种强白盒攻击（GCG、BEAST）及三种黑盒基准测试，采用SAE增强的模型相较未防护基线，越狱成功率最高降低5倍，并显著减少跨模型攻击迁移性。参数消融实验表明：(i) L0稀疏度与攻击成功率呈单调递减关系；(ii) 层级依赖的防御-性能权衡存在，中间层能较好平衡鲁棒性与正常表现。这些结果支持表示瓶颈假说：稀疏投影重构了越狱攻击所依赖的优化几何结构。
原文摘要 · Abstract (English)
Large Language Models (LLMs) remain vulnerable to optimization-based jailbreak attacks that exploit internal gradient structure. While Sparse Autoencoders (SAEs) are widely used for interpretability, their robustness implications remain underexplored. We present a study of integrating pretrained SAEs into transformer residual streams at inference time, without modifying model weights or blocking gradients. Across four model families (Gemma, LLaMA, Mistral, Qwen) and two strong white-box attacks (GCG, BEAST) plus three black-box benchmarks, SAE-augmented models achieve up to a 5x reduction in jailbreak success rate relative to the undefended baseline and reduce cross-model attack transferability. Parametric ablations reveal (i) a monotonic dose-response relationship between L0 sparsity and attack success rate, and (ii) a layer-dependent defense-utility tradeoff, where intermediate layers balance robustness and clean performance. These findings are consistent with a representational bottleneck hypothesis: sparse projection reshapes the optimization geometry exploited by jailbreak attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。