小模型也能精准识别有害指令，且结果稳定可靠。
Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families
- 用轻量MLP探测器分析大模型最后一层激活值，识别有害提示。
- 在多个模型上复现效果，F1分数与原论文差距小于0.37个百分点。
- 相同输入下不同随机种子结果一致，说明输出高度稳定。
Khatri等（2026）[DOI: 10.1109/DSN-W70714.2026.00027]表明，针对单个80亿参数模型（LLaMA-3.1-8B）最后层激活值的轻量级MLP探测器，在使用每个基准一个探测器的情况下，其检测有害提示的F1分数可媲美大1000倍的守卫模型。我们端到端复现该流程，并沿原始研究未覆盖的两个方向扩展：第一，测试该结果是否在其他架构和规模模型中通用，对Gemma-4-E4B、Mistral-7B-v0.3和Qwen2-7B等模型训练相同探测器，使用WildJailbreak、BeaverTails和AEGIS 2.0三个基准；第二，测试推理过程中的非确定性对性能的影响，通过五组随机种子重复提取并测量F1分数方差。结果表明，对原始LLaMA模型的基准复现精度在0.37个百分点内（BeaverTails上为0.2点以内）。所测试的其他模型家族的探测器性能也保持在原报告值1个点以内。改变随机种子的实验发现，所有测试架构的最终令牌潜在向量均保持不变，表明结果具有高度稳定性。
原文摘要 · Abstract (English)
Khatri et al. (2026) [DOI: 10.1109/DSN-W70714.2026.00027] show that lightweight MLP probes on final-layer activations of a single 8B model (LLaMA-3.1-8B) detect harmful prompts at F1 competitive with guard models 1000x larger, using one probe per benchmark. We reproduce this pipeline end-to-end and extend it along two axes the original study leaves open. First, we test whether the result generalizes across other model architecture and scale by training identical probes on activations from models like Gemma-4-E4B, Mistral-7B-v0.3, and Qwen2-7B, using the three benchmarks (WildJailbreak, BeaverTails, AEGIS 2.0). Second, we test how much of the reported performance is affected by non-determinism during inference by repeating extraction under five random seeds and measuring the variance of F1 scores. Our results reproduce the original LLaMA model benchmarks within 0.37 percentage points of the original F1 scores (and within 0.2 points on BeaverTails). We find that the original MLP probe architecture extends to other model families with F1 scores within a point of the values reported for LLaMA-3.1-8B. Our experiments varying seed values reveal an interesting observation: final token latent vectors remained the same for all tested architectures irrespective of the seed values used.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。