定位大模型越狱攻击的脆弱点,发现只需单个关键词就能触发漏洞。
From Concept-Aligned Tokens to Vulnerable Features: Mechanistic Localization of Jailbreaks
- 通过稀疏自编码器分析模型内部表示,用提示词定位有害特征子群。
- 仅用单个有害提示词即可识别出与越狱相关的特征组,效果接近全量聚类。
- 漏洞特征集中在中后期层,说明攻击可精准定向于模型特定部分。
越狱攻击暴露了安全对齐大模型的一个顽固缺陷:模型可能被诱导产生有害行为,但其内部表征的脆弱位置尚不明确。现有机制安全研究多以全局拒绝方向、激活操控向量或拒绝相关SAE特征等宏观对象解释该现象。本文提出新思路:是否能追踪到更细粒度、由提示词决定的SAE特征子群?我们构建了一条基于令牌的机制分析流程,将Gemma-2-2B的残差流分解为SAE特征,并识别与有害行为相关的特征子群。利用BeaverTails中的单类别有害样本减少跨类别干扰,提取对抗响应中的有害概念,并通过子空间相似性将其与相关提示词对齐。采用三种分组策略——基于聚类、层次链接和单令牌驱动——在全部26层中识别SAE特征子群。最后放大每组顶峰特征,由标准化有害性评估器测试生成结果。结果显示,单令牌驱动分组在有害性表现上与全量聚类相当,表明仅凭单一有害提示词即可定位与漏洞相关的特征子群,无需依赖整体聚类聚合。这些子群分布于早期及中晚期层,且在中晚期层集中更强,说明针对性操控可暴露具体模型弱点。总体表明,越狱敏感性可归因于稀疏、令牌局部化的SAE特征子群,补充了以往基于广泛对抗、拒绝或操控方向的解释。
原文摘要 · Abstract (English)
Jailbreak attacks expose a persistent failure mode in safety-aligned LLMs: models can be pushed into harmful behavior, but the internal representations enabling this shift remain poorly localized. Recent mechanistic safety studies often explain such behavior through broad representational objects, including global refusal directions, activation steering vectors, and refusal-related SAE features. We instead ask whether jailbreak vulnerability can be traced to finer-grained, prompt-conditioned SAE feature subgroups. We introduce a token-driven mechanistic pipeline that decomposes the residual stream of Gemma-2-2B into Sparse Autoencoder (SAE) features and identifies feature subgroups associated with unsafe behavior. Using single-category unsafe examples from BeaverTails to reduce cross-category interference, we extract harmful concepts from adversarial responses and align them with concept-relevant prompt tokens through subspace similarity. We then apply three feature-grouping strategies: cluster-based, hierarchical-linkage, and single-token-driven, to identify SAE feature subgroups across all 26 layers. Finally, we amplify the top features in each subgroup and evaluate the resulting generations with a standardized harmfulness judge. Single-token-driven grouping achieves harmfulness comparable to full cluster-based grouping, showing that individual harmful prompt tokens are sufficient to localize vulnerability-relevant SAE feature subgroups without relying on broader cluster-level aggregation. These subgroups appear across early and mid-to-late layers, with stronger concentration in mid-to-late layers, where targeted steering exposes specific model vulnerabilities. Overall, our results suggest that jailbreak susceptibility can be traced to sparse, token-localized SAE feature subgroups, complementing prior accounts based on broad adversarial, refusal, or steering directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。