通过分析注意力分数分布，实时检测大模型幻觉。
AggTruth: Contextual Hallucination Detection using Aggregated Attention Scores in LLMs
- 利用上下文注意力分数的分布模式识别幻觉。
- 在多种模型和任务中均超越现有最优方法。
- 精选注意力头能显著提升检测效果，适合模型部署者。
在实际应用中，大语言模型（LLMs）即使在检索增强生成（RAG）设置下仍会产生幻觉，严重影响其部署。本文提出AggTruth，一种基于内部注意力分数分布的在线上下文幻觉检测方法。我们设计了四种不同聚合策略的变体，适用于各类大模型。实验表明，AggTruth在同任务与跨任务场景中均表现稳定，且在多个测试中优于当前最先进方法。我们还深入分析了特征选择技术，并验证了所选注意力头数量对检测性能的影响，证明精心挑选注意力头对获得最佳结果至关重要。
原文摘要 · Abstract (English)
In real-world applications, Large Language Models (LLMs) often hallucinate, even in Retrieval-Augmented Generation (RAG) settings, which poses a significant challenge to their deployment. In this paper, we introduce AggTruth, a method for online detection of contextual hallucinations by analyzing the distribution of internal attention scores in the provided context (passage). Specifically, we propose four different variants of the method, each varying in the aggregation technique used to calculate attention scores. Across all LLMs examined, AggTruth demonstrated stable performance in both same-task and cross-task setups, outperforming the current SOTA in multiple scenarios. Furthermore, we conducted an in-depth analysis of feature selection techniques and examined how the number of selected attention heads impacts detection performance, demonstrating that careful selection of heads is essential to achieve optimal results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。