arXiv:2603.27412cs.LGcs.AI2026-03被引 3

通过分析模型残差流角度偏差,无训练检测有害提示。

The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams

  • 用正常提示的激活主成分建参考方向,测新提示的角度偏离。
  • 六种模型均达AUROC≥0.937,检测有害与良性攻击提示准确率1.000。
  • 不依赖有害数据训练,适合安全评估与模型对齐研究者。

我们提出LatentBiopsy,一种无需训练的有害提示检测方法,通过分析大语言模型残差流激活的几何特性实现。基于200个安全规范提示,计算目标层激活的主成分方向,以新提示与该方向的径向夹角θ作为异常评分依据。异常得分定义为θ在正态分布拟合下的负对数似然,对称地标记所有方向的偏离。无需任何有害样本训练。我们在Qwen3.5-0.8B和Qwen2.5-0.5B两个模型族的三种变体(基础、指令微调、拒绝机制剔除)上评估:六种配置下,有害与正常提示检测的AUROC均≥0.937,有害与良性攻击提示区分的AUROC达1.000,单次查询开销低于毫秒级。三个发现:第一,拒绝机制剔除后几何特征仍保留,两组剔除版本与指令微调版的AUROC差距≤0.015,证明有害意图表征与生成拒绝机制在几何上解耦;第二,有害提示的角度分布极窄(σ_θ≈0.03弧度),较正常分布(σ_θ≈0.27弧度)紧密一个数量级,且在所有对齐阶段均保持不变;第三,同一深度下,Qwen3.5-0.8B中危害提示位于外环,而Qwen2.5-0.5B中位于内环,直接支持方向无关评分规则的设计。

原文摘要 · Abstract (English)

We present LatentBiopsy, a training-free method for detecting harmful prompts by analysing the geometry of residual-stream activations in large language models. Given 200 safe normative prompts, LatentBiopsy computes the leading principal component of their activations at a target layer and characterises new prompts by their radial deviation angle $θ$ from this reference direction. The anomaly score is the negative log-likelihood of $θ$ under a Gaussian fit to the normative distribution, flagging deviations symmetrically regardless of orientation. No harmful examples are required for training. We evaluate two complete model triplets from the Qwen3.5-0.8B and Qwen2.5-0.5B families: base, instruction-tuned, and \emph{abliterated} (refusal direction surgically removed via orthogonalisation). Across all six variants, LatentBiopsy achieves AUROC $\geq$0.937 for harmful-vs-normative detection and AUROC = 1.000 for discriminating harmful from benign-aggressive prompts (XSTest), with sub-millisecond per-query overhead. Three empirical findings emerge. First, geometry survives refusal ablation: both abliterated variants achieve AUROC at most 0.015 below their instruction-tuned counterparts, establishing a geometric dissociation between harmful-intent representation and the downstream generative refusal mechanism. Second, harmful prompts exhibit a near-degenerate angular distribution ($σ_θ\approx 0.03$ rad), an order of magnitude tighter than the normative distribution ($σ_θ\approx 0.27$ rad), preserved across all alignment stages including abliteration. Third, the two families exhibit opposite ring orientations at the same depth: harmful prompts occupy the outer ring in Qwen3.5-0.8B but the inner ring in Qwen2.5-0.5B, directly motivating the direction-agnostic scoring rule.

异常检测语言模型几何分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。