通过分析注意力拓扑，无需训练即可高效检测大模型工具使用幻觉。
Spectral Guardrails for Agents in the Wild: Detecting Tool Use Hallucinations via Attention Topology
- 基于注意力拓扑的谱分析，不依赖标注数据构建安全防护机制。
- 单层特征检测率达98.2%（213/217），误报率低且阈值单一。
- 适用于部署在真实环境中的自主智能体，尤其适合提升安全性。
将自主智能体部署于真实环境需要可靠的工具使用失败防护机制。本文提出一种无需训练的防护框架，基于注意力拓扑的谱分析，可补充监督式方法。在 Llama 3.1 8B 上，多特征检测实现 97.7% 召回率，平衡部署下召回率为 86.1%，精度达 81.0%。最显著发现是：单层谱特征即可近乎完美检测幻觉——Llama L26 Smoothness 达到 98.2% 召回率（213/217 幻觉被捕捉），Mistral L3 Entropy 达到 94.7% 召回率。这表明幻觉不仅是错误输出，更是模型注意力状态的热力学转变：出错时注意力趋于噪声。在相同领域（$N=1000$, $T=0.3$，通用域，幻觉率 20–22%）的跨模型控制实验中，揭示“喧嚣说谎者”现象：Llama 3.1 8B 的错误在谱空间中表现为灾难性变化，极易检测；而 Mistral 7B 表现最佳（AUC 0.900）。这些发现确立了谱分析作为智能体安全的原理性、高效框架。
原文摘要 · Abstract (English)
Deploying autonomous agents in the wild requires reliable safeguards against tool use failures. We propose a training free guardrail based on spectral analysis of attention topology that complements supervised approaches. On Llama 3.1 8B, our method achieves 97.7\% recall with multi-feature detection and 86.1\% recall with 81.0\% precision for balanced deployment, without requiring any labeled training data. Most remarkably, we discover that single layer spectral features act as near-perfect hallucination detectors: Llama L26 Smoothness achieves 98.2\% recall (213/217 hallucinations caught) with a single threshold, and Mistral L3 Entropy achieves 94.7\% recall. This suggests hallucination is not merely a wrong token but a thermodynamic state change: the model's attention becomes noise when it errs. Through controlled cross-model evaluation on matched domains ($N=1000$, $T=0.3$, same General domain, hallucination rates 20--22\%), we reveal the ``Loud Liar'' phenomenon: Llama 3.1 8B's failures are spectrally catastrophic and dramatically easier to detect, while Mistral 7B achieves the best discrimination (AUC 0.900). These findings establish spectral analysis as a principled, efficient framework for agent safety.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。