通过分析注意力图拓扑差异，高效检测大模型幻觉输出
Hallucination Detection in LLMs with Topological Divergence on Attention Graphs
- 基于注意力矩阵构建图结构，用拓扑差异度量判断幻觉
- 特定注意力头的拓扑差异越高，越容易产生幻觉内容
- 无需大量标注数据，适合在检索增强生成中部署
幻觉，即生成事实性错误内容，仍是大语言模型（LLMs）面临的关键挑战。我们提出TOHA——一种基于拓扑结构的幻觉检测方法，适用于检索增强生成（RAG）场景。该方法利用拓扑差异度量，量化由注意力矩阵诱导出的图结构的结构性质。分析提示词与响应子图之间的拓扑差异发现：特定注意力头的拓扑差异值越高，越容易生成幻觉输出，且该模式不依赖于具体数据集。大量实验涵盖问答与摘要任务，结果表明该方法在多个基准上达到或接近当前最优性能，同时仅需极少标注数据与计算资源。研究结果表明，分析注意力矩阵的拓扑结构可作为大模型事实可靠性的一种高效且鲁棒的指示器。
原文摘要 · Abstract (English)
Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models (LLMs). We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting, which leverages a topological divergence metric to quantify the structural properties of graphs induced by attention matrices. Examining the topological divergence between prompt and response subgraphs reveals consistent patterns: higher divergence values in specific attention heads correlate with hallucinated outputs, independent of the dataset. Extensive experiments - including evaluation on question answering and summarization tasks - show that our approach achieves state-of-the-art or competitive results on several benchmarks while requiring minimal annotated data and computational resources. Our findings suggest that analyzing the topological structure of attention matrices can serve as an efficient and robust indicator of factual reliability in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。