arXiv:2511.05804cs.LGcs.SY2025-11被引 1

用注意力图谱的频谱特征提前发现推理错误,毫秒级拦截污染

Catching Contamination Before Generation: Spectral Kill Switches for Agents

  • 通过注意力机制生成的词元图谱,计算高频能量比与谱熵
  • 高频能量比在多模型上呈现双峰分布,可实现毫秒级判断
  • 无需训练,可嵌入检索增强系统作实时安全监控

代理语言模型执行多步推理时,中间步骤可能因上下文不一致、检索错误或对抗输入而被污染,事后评估已无法挽回。本文提出一种无需额外训练的诊断方法,仅通过前向传播即可在代理运行时输出接受或拒绝信号。该方法分析注意力机制生成的词元图谱,在早期层计算两个谱统计量:高频能量比与谱熵。我们形式化了这些信号,建立不变性,并提供有限样本估计器与不确定性量化。在双模式混合假设下,具备单调似然比性质,单一阈值的高频能量比在贝叶斯意义上最优,用于检测上下文不一致性。实验表明,高频能量比在多个模型家族的上下文验证中表现出稳健的双峰特性,可在硬件上实现低于1毫秒的开销。我们展示了其在检索增强代理流程中的集成,并讨论作为内联安全监控的部署方案。该方法能在模型处理文本过程中提前发现污染,防止错误进入推理链。

原文摘要 · Abstract (English)

Agentic language models compose multi step reasoning chains, yet intermediate steps can be corrupted by inconsistent context, retrieval errors, or adversarial inputs, which makes post hoc evaluation too late because errors propagate before detection. We introduce a diagnostic that requires no additional training and uses only the forward pass to emit a binary accept or reject signal during agent execution. The method analyzes token graphs induced by attention and computes two spectral statistics in early layers, namely the high frequency energy ratio and spectral entropy. We formalize these signals, establish invariances, and provide finite sample estimators with uncertainty quantification. Under a two regime mixture assumption with a monotone likelihood ratio property, we show that a single threshold on the high frequency energy ratio is optimal in the Bayes sense for detecting context inconsistency. Empirically, the high frequency energy ratio exhibits robust bimodality during context verification across multiple model families, which enables gating decisions with overhead below one millisecond on our hardware and configurations. We demonstrate integration into retrieval augmented agent pipelines and discuss deployment as an inline safety monitor. The approach detects contamination while the model is still processing the text, before errors commit to the reasoning chain.

大模型安全在线检测谱分析推理鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。