通过跨层注意力探针检测大模型幻觉，提升生成可靠性。
Cross-Layer Attention Probing for Fine-Grained Hallucination Detection
- 将模型全残差流激活联合建模，捕捉跨层注意力模式。
- 在五种模型三类任务中优于基线，可区分不同采样结果的幻觉。
- 适合关注生成质量与可靠性的研究人员和应用开发者。
随着大规模语言模型（LLMs）在各类应用中的普及，其生成不准确内容（即幻觉）的问题日益引发可靠性担忧。本文提出跨层注意力探针（CLAP），一种新型激活探针技术，将模型整个残差流的激活作为联合序列进行处理。在五个LLM和三个任务上的实证评估表明，相比基线方法，CLAP在贪婪解码及高温度采样响应中均提升了幻觉检测能力，实现细粒度检测——即能够区分同一提示下不同采样结果中的幻觉与非幻觉内容。基于此，我们提出先检测后缓解的策略，相比直接缓解方法，显著降低幻觉并提升模型可靠性。此外，实验显示CLAP在分布外场景下仍保持高可靠性。
原文摘要 · Abstract (English)
With the large-scale adoption of Large Language Models (LLMs) in various applications, there is a growing reliability concern due to their tendency to generate inaccurate text, i.e. hallucinations. In this work, we propose Cross-Layer Attention Probing (CLAP), a novel activation probing technique for hallucination detection, which processes the LLM activations across the entire residual stream as a joint sequence. Our empirical evaluations using five LLMs and three tasks show that CLAP improves hallucination detection compared to baselines on both greedy decoded responses as well as responses sampled at higher temperatures, thus enabling fine-grained detection, i.e. the ability to disambiguate hallucinations and non-hallucinations among different sampled responses to a given prompt. This allows us to propose a detect-then-mitigate strategy using CLAP to reduce hallucinations and improve LLM reliability compared to direct mitigation approaches. Finally, we show that CLAP maintains high reliability even when applied out-of-distribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。