用稀疏自编码器定位大模型幻觉,提升RAG生成可信度。
Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders
- 通过稀疏自编码器分解模型内部激活,识别幻觉触发特征。
- 检测准确率优于现有方法,且推理成本低。
- 提供可解释理由,适合需高可信生成的场景。
检索增强生成(RAG)通过引用外部证据提升大语言模型(LLM)的准确性,但生成内容与源文档矛盾或超出其范围的问题仍普遍存在。现有幻觉检测方法依赖大规模标注数据训练或调用外部LLM评判,成本高昂;部分利用模型内部表示的方法准确率不足。受可解释性研究启发,本文采用稀疏自编码器(SAEs)分离模型内部激活,成功识别出在RAG幻觉中特异性激活的特征。基于信息论特征选择与可加特征建模的系统化流程,提出轻量级检测器RAGLens,仅使用模型内部表示即可精准标记不忠实输出。RAGLens不仅性能超越现有方法,还提供可解释决策依据,支持事后修正。最后,验证设计合理性,并揭示幻觉信号在模型中的分布规律。代码已开源。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) improves the factuality of large language models (LLMs) by grounding outputs in retrieved evidence, but faithfulness failures, where generations contradict or extend beyond the provided sources, remain a critical challenge. Existing hallucination detection methods for RAG often rely either on large-scale detector training, which requires substantial annotated data, or on querying external LLM judges, which leads to high inference costs. Although some approaches attempt to leverage internal representations of LLMs for hallucination detection, their accuracy remains limited. Motivated by recent advances in mechanistic interpretability, we employ sparse autoencoders (SAEs) to disentangle internal activations, successfully identifying features that are specifically triggered during RAG hallucinations. Building on a systematic pipeline of information-based feature selection and additive feature modeling, we introduce RAGLens, a lightweight hallucination detector that accurately flags unfaithful RAG outputs using LLM internal representations. RAGLens not only achieves superior detection performance compared to existing methods, but also provides interpretable rationales for its decisions, enabling effective post-hoc mitigation of unfaithful RAG. Finally, we justify our design choices and reveal new insights into the distribution of hallucination-related signals within LLMs. The code is available at https://github.com/Teddy-XiongGZ/RAGLens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。