arXiv:2605.00269cs.CLcs.LG2026-05

发现大模型检测异常输入时受序列长度干扰,提出双路径新框架提升检测可靠性。

How Language Models Process Out-of-Distribution Inputs: A Two-Pathway Framework

  • 分离语义嵌入与处理轨迹,分别捕捉主题偏移和隐性意图输入
  • 在词汇重叠的隐蔽攻击上达到0.850 AUROC,优于传统方法
  • 适用于安全检测、对抗样本防御等需识别隐蔽异常的场景

当前白盒异常输入检测方法(如CED、RAUQ、WildGuard)看似有效,但实则受输入长度(|r| ≥ 0.61)结构性干扰,在长度匹配评估下性能趋近随机。即使原始注意力熵(各头与层均值H(α))也存在相同问题,根源在于注意力对输入长度呈Theta(log T)依赖。为消除混淆,我们提出双路径框架:嵌入表征内容主题(擅长主题变化),处理轨迹(隐藏状态跨层演化)捕捉模型处理过程。两者能力随词汇透明度变化:嵌入法在词汇显著不同的异常中表现优,轨迹特征可检测共享词汇的隐蔽输入(平均AUROC 0.721;越狱攻击达0.850)。三重证据支持:(1)6项任务中k-NN与轨迹评分出现交叉胜出,路径依异常类型而异;(2)逐层分析显示,第0层k-NN信号几乎全为长度伪影(越狱攻击:0.759→0.389);(3)电路归因表明对抗任务更激活注意力回路(p=0.022;越狱修复测试p<0.001),跨模型部分复现。代码将于发表后公开。

原文摘要 · Abstract (English)

Recent white-box OOD detection methods for LLMs -- including CED, RAUQ, and WildGuard confidence scores -- appear effective, but we show they are structurally confounded by sequence length (|r| >= 0.61) and collapse to near-chance under length-matched evaluation. Even raw attention entropy (mean H(alpha) across heads and layers), a natural baseline we include for completeness, shows the same confound. The confound stems from attention's Theta(log T) dependence on input length. To identify genuine OOD signals after deconfounding, we propose a two-pathway framework: embeddings capture what text is about (effective for topic shifts), while the processing trajectory -- hidden-state evolution across layers -- captures how the model processes input. The relative power of each pathway varies along a vocabulary-transparency spectrum: embedding methods excel on vocabulary-distinctive OOD, while trajectory features detect covert-intent inputs that share vocabulary with normal text (0.721 avg AUROC; Jailbreak: 0.850). Three evidence lines support this framework: (1) a crossover between k-NN and trajectory scoring across 6 tasks, where each pathway wins on different OOD types; (2) a per-layer analysis showing that layer-0 k-NN signal is almost entirely a length artifact (Jailbreak: 0.759 raw -> 0.389 matched) -- processing constructs genuine OOD signal from near-chance embeddings; and (3) circuit attribution showing adversarial tasks engage attention circuits more than semantic tasks (p = 0.022; Jailbreak patching p < 0.001), with partial cross-model replication. Code release upon publication.

大模型安全异常检测注意力机制鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。