arXiv:2504.17004cs.LGcs.AI2025-04被引 14

理论上,仅靠正确语句无法自动检测大模型幻觉。

(Im)possibility of Automated Hallucination Detection in Large Language Models

  • 将幻觉检测转化为语言识别问题,揭示其根本局限性
  • 仅用正确样本训练时,多数情况下检测不可能
  • 引入专家标注的错误样本后,检测变为可行

本文提出一个理论框架,分析大语言模型(LLMs)幻觉自动检测的可能性。受经典的Gold-Angluin语言识别框架启发,研究发现幻觉检测与语言识别等价:若仅用目标语言中的正确例句训练检测器,则对大多数语言集合而言检测本质上不可行。然而,当训练数据包含专家标注的错误语句(负样本)时,检测在所有可数语言集合上均可实现。该结果强调了专家反馈在训练幻觉检测器中的核心作用,为基于人类反馈的强化学习(RLHF)等方法提供了理论支持。

原文摘要 · Abstract (English)

Is automated hallucination detection possible? In this work, we introduce a theoretical framework to analyze the feasibility of automatically detecting hallucinations produced by large language models (LLMs). Inspired by the classical Gold-Angluin framework for language identification and its recent adaptation to language generation by Kleinberg and Mullainathan, we investigate whether an algorithm, trained on examples drawn from an unknown target language $K$ (selected from a countable collection) and given access to an LLM, can reliably determine whether the LLM's outputs are correct or constitute hallucinations. First, we establish an equivalence between hallucination detection and the classical task of language identification. We prove that any hallucination detection method can be converted into a language identification method, and conversely, algorithms solving language identification can be adapted for hallucination detection. Given the inherent difficulty of language identification, this implies that hallucination detection is fundamentally impossible for most language collections if the detector is trained using only correct examples from the target language. Second, we show that the use of expert-labeled feedback, i.e., training the detector with both positive examples (correct statements) and negative examples (explicitly labeled incorrect statements), dramatically changes this conclusion. Under this enriched training regime, automated hallucination detection becomes possible for all countable language collections. These results highlight the essential role of expert-labeled examples in training hallucination detectors and provide theoretical support for feedback-based methods, such as reinforcement learning with human feedback (RLHF), which have proven critical for reliable LLM deployment.

幻觉检测语言识别强化学习理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。