arXiv:2604.15945cs.CLcs.LG2026-04中稿 · IJCNN 2026被引 1

通过内置检测头联合训练,减少大模型生成幻觉。

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

论文配图:RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
图 1 · 摘自论文原文
  • 在大模型中加入轻量级幻觉检测头,实现生成与检测联合优化。
  • 在多个基准上实现顶尖幻觉检测效果,生成幻觉率显著降低。
  • 适合关注生成可靠性、需降低幻觉风险的研究与应用者。

检索增强生成(RAG)广泛用于为大语言模型(LLMs)注入外部信息,如最新或领域特定知识。然而,现有模型仍会产生封闭域幻觉,生成内容缺乏检索到的上下文支持。当前检测方法通常将幻觉视为事后问题,依赖黑箱一致性检查或对冻结内部表示的探针。本文证明,基于内部状态表示的幻觉检测可作为直接训练信号。我们提出 RAGognize,一个包含细粒度标注的自然发生封闭域幻觉数据集,以及 RAGognizer,一种集成轻量检测头的幻觉感知微调方法,使语言建模与幻觉检测联合优化。该联合目标促使模型提升内部状态在幻觉上的可分性,同时学习生成语法正确且有意义的回答。在多个基准测试中,RAGognizer 在令牌级幻觉检测上达到最先进水平,并显著降低生成过程中的幻觉率,且不损害语言质量或相关性。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) is widely used to augment the input to Large Language Models (LLMs) with external information, such as recent or domain-specific knowledge. Nonetheless, current models still produce closed-domain hallucinations and generate content that is unsupported by the retrieved context. Current detection approaches typically treat hallucination as a post-hoc problem, relying on black-box consistency checks or probes over frozen internal representations. In this work, we demonstrate that hallucination detection based on internal state representation can also serve as a direct training signal. We introduce RAGognize, a dataset of naturally occurring closed-domain hallucinations with token-level annotations, and RAGognizer, a hallucination-aware fine-tuning approach that integrates a lightweight detection head into an LLM, allowing for the joint optimization of language modeling and hallucination detection. This joint objective forces the model to improve the separability of its internal states regarding hallucinations while simultaneously learning to generate well-formed and meaningful responses. Across multiple benchmarks, RAGognizer achieves state-of-the-art token-level hallucination detection while substantially reducing hallucination rates during generation, without degrading language quality or relevance.

幻觉检测RAG微调大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。