通过因果图注意力分析，降低大模型幻觉率。
Unmasking Hallucinations: A Causal Graph-Attention Perspective on Factual Reliability in Large Language Models
- 构建词元级图网络，融合注意力与梯度影响评分
- 提出因果贡献分，量化每个词元的事实依赖度
- 动态重加权减少幻觉节点影响，适合医疗法律场景
本文聚焦于大语言模型（LLMs）产生的幻觉问题。尽管LLMs具备出色的语言理解与生成能力，但其输出常包含事实错误、误导性或无依据的内容，尤其在医疗诊断与法律推理等关键场景中引发严重风险。为此，本文提出因果图注意力网络（GCAN）框架，通过构建词元级图结构，结合自注意力权重与基于梯度的影响得分，解析Transformer内部的注意力流动。我们引入新的因果贡献分（CCS）量化每个词元的事实依赖程度，并设计事实锚定的图重加权层，在生成过程中动态削弱易产生幻觉节点的影响。在TruthfulQA和HotpotQA等标准基准上的实验表明，该方法使幻觉率降低27.8%，事实准确率相比基线检索增强生成（RAG）模型提升16.4%。本工作提升了LLM的可解释性、鲁棒性与事实可靠性。
原文摘要 · Abstract (English)
This paper primarily focuses on the hallucinations caused due to AI language models(LLMs).LLMs have shown extraordinary Language understanding and generation capabilities .Still it has major a disadvantage hallucinations which give outputs which are factually incorrect ,misleading or unsupported by input data . These hallucinations cause serious problems in scenarios like medical diagnosis or legal reasoning.Through this work,we propose causal graph attention network (GCAN) framework that reduces hallucinations through interpretation of internal attention flow within a transformer architecture with the help of constructing token level graphs that combine self attention weights and gradient based influence scores.our method quantifies each tokens factual dependency using a new metric called the Causal Contribution Score (CCS). We further introduce a fact-anchored graph reweighting layer that dynamically reduces the influence of hallucination prone nodes during generation. Experiments on standard benchmarks such as TruthfulQA and HotpotQA show a 27.8 percent reduction in hallucination rate and 16.4 percent improvement in factual accuracy over baseline retrieval-augmented generation (RAG) models. This work contributes to the interpretability,robustness, and factual reliability of future LLM architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。