通过解耦外部与内部知识使用，精准识别RAG模型幻觉。
ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
- 分析LLM在RAG中对内外知识的利用机制，定位幻觉根源。
- 在多个数据集上检测准确率显著提升，最高达89.7%。
- 适合关注大模型可解释性与生成可信度的研究者。
检索增强生成（RAG）模型旨在引入外部知识以减少因参数化知识不足导致的幻觉。然而，即使检索内容准确相关，RAG模型仍可能生成与检索信息矛盾的输出。检测此类幻觉需厘清大语言模型（LLMs）如何利用外部与参数化知识。现有方法通常仅关注其中一种机制，或未能分离二者交织影响，导致检测不准确。本文研究了RAG场景下幻觉的内部机制，发现当知识前馈神经网络（Knowledge FFNs）在残差流中过度强调参数化知识，而复制头（Copying Heads）未能有效保留或整合检索内容时，幻觉便会产生。基于此，我们提出ReDeEP，一种通过解耦外部上下文与参数化知识利用来检测幻觉的新方法。实验表明，ReDeEP显著提升了幻觉检测精度。此外，我们还提出AARF，通过调节知识前馈神经网络与复制头的贡献，缓解幻觉问题。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) models are designed to incorporate external knowledge, reducing hallucinations caused by insufficient parametric (internal) knowledge. However, even with accurate and relevant retrieved content, RAG models can still produce hallucinations by generating outputs that conflict with the retrieved information. Detecting such hallucinations requires disentangling how Large Language Models (LLMs) utilize external and parametric knowledge. Current detection methods often focus on one of these mechanisms or without decoupling their intertwined effects, making accurate detection difficult. In this paper, we investigate the internal mechanisms behind hallucinations in RAG scenarios. We discover hallucinations occur when the Knowledge FFNs in LLMs overemphasize parametric knowledge in the residual stream, while Copying Heads fail to effectively retain or integrate external knowledge from retrieved content. Based on these findings, we propose ReDeEP, a novel method that detects hallucinations by decoupling LLM's utilization of external context and parametric knowledge. Our experiments show that ReDeEP significantly improves RAG hallucination detection accuracy. Additionally, we introduce AARF, which mitigates hallucinations by modulating the contributions of Knowledge FFNs and Copying Heads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。