发现大模型安全检测在最后令牌读取时会漏检早期隐藏信息，导致越狱攻击失败。
Before the Last Token: Diagnosing Final-Token Safety Probe Failures
- 通过分析提示词前缀阶段的隐藏状态，发现安全探针只看最后一刻状态会漏掉早期危险信号。
- 在三个指令微调模型上，探针对干净有害提示召回率高，但对越狱攻击漏检率超40%，误报也多。
- 提出基于PCA-HMM的轨迹模型，能从完整序列中恢复被漏掉的危险信号，适合安全审计使用。
最终令牌安全探针仅在提示词预填充后读取单一隐藏状态，但越狱提示中的危险证据常分散在早期用户令牌表示中，无法被该读取方式捕获。本文使用仅在清洁有害与良性提示上训练的SafeSwitch风格探针，评估其在三个指令微调大模型上的表现。探针对清洁有害提示具有高召回率，但显著漏检越狱攻击，并在安全邻近的良性提示上产生大量误报。子空间分析表明，被漏检的越狱攻击与良性提示在探针表示子空间中难以区分的方向上存在差异，增加探针瓶颈宽度也无法可靠缓解此不匹配。逐令牌前缀分析显示，探针可见的危险信号通常出现在序列早期，但在最终令牌读取时未被暴露；而朴素的全局最大池化则在安全提示上过度触发。一个仅在相同清洁数据集上训练的简单PCA-HMM轨迹模型，无需额外标注即可从用户内容前缀轨迹中恢复多数最终令牌漏检案例，且无传统池化带来的灾难性误报行为，提示应将轨迹感知的隐藏状态分析作为最终令牌探针的诊断补充。
原文摘要 · Abstract (English)
Final-token safety probes monitor a single hidden state after prompt prefill, but jailbreak prompts can contain probe-visible unsafe evidence distributed across earlier user-token representations that is missed by this readout. We study this prefill-time failure mode using SafeSwitch-style probes trained only on clean harmful and benign prompts across three instruction-tuned LLMs. The probes achieve high recall on clean harmful prompts, but miss many jailbreaks and can produce false positives on safety-adjacent benign prompts. Subspace analyses suggest that missed jailbreaks differ from clean benign prompts along directions that are poorly captured by the probe's representational subspace, and increasing probe bottleneck width does not reliably resolve this mismatch. Token-level prefill analyses reveal that probe-visible unsafe evidence often appears earlier in the sequence but is not exposed at the final-token readout, while naive max-pooling over token positions overfires on safe prompts. A simple PCA-HMM trajectory model, trained only on the same clean split, recovers many final-token misses from user-content prefill trajectories without the catastrophic false-positive behavior of naive token pooling, motivating trajectory-aware hidden-state analyses as diagnostic complements to final-token probes
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。