arXiv:2506.17748cs.CLcs.AI2025-06被引 3

通过分析模型内部表示的解耦程度,单次运行即可高效检测语言模型幻觉。

HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations

  • 利用隐藏层表示间的统计解耦度衡量幻觉风险,无需多轮生成。
  • 在6个不同规模模型上平均提升29%的检测准确率(AUC-ROC)。
  • 计算耗时仅为现有方法的49%,适合实际部署场景。

当前语言模型虽流畅,但常产生与事实不符或偏离输入语境的内容,即“幻觉”,严重影响可靠性。现有检测方法多依赖多次生成,成本高。本文提出HIDE(Hallucination Detection via Decoupled Representations),一种单次运行、无需训练的检测方法。其核心思想是:幻觉源于模型对输入上下文与输出之间的内部表示解耦。通过希尔伯特-施密特独立性准则(HSIC)量化生成过程中隐藏状态表示的解耦程度。在四个问答数据集上评估六种不同规模的开源模型,结果表明,HIDE在几乎所有设置下均优于其他单次运行方法,平均相对提升约29%的AUC-ROC;同时,性能接近甚至超越多轮生成的先进方法,平均提升约3%,且计算时间减少约51%。研究证实,利用模型内部表示解耦可实现高效实用的幻觉检测。

原文摘要 · Abstract (English)

Contemporary Language Models (LMs), while impressively fluent, often generate content that is factually incorrect or unfaithful to the input context - a critical issue commonly referred to as 'hallucination'. This tendency of LMs to generate hallucinated content undermines their reliability, especially because these fabrications are often highly convincing and therefore difficult to detect. While several existing methods attempt to detect hallucinations, most rely on analyzing multiple generations per input, leading to increased computational cost and latency. To address this, we propose a single-pass, training-free approach for effective Hallucination detectIon via Decoupled rEpresentations (HIDE). Our approach leverages the hypothesis that hallucinations result from a statistical decoupling between an LM's internal representations of input context and its generated output. We quantify this decoupling using the Hilbert-Schmidt Independence Criterion (HSIC) applied to hidden-state representations extracted while generating the output sequence. We conduct extensive experiments on four diverse question answering datasets, evaluating both faithfulness and factuality hallucinations across six open-source LMs of varying scales and properties. Our results demonstrate that HIDE outperforms other single-pass methods in almost all settings, achieving an average relative improvement of ~29% in AUC-ROC over the best-performing single-pass strategy across various models and datasets. Additionally, HIDE shows competitive and often superior performance with multi-pass state-of-the-art methods, obtaining an average relative improvement of ~3% in AUC-ROC while consuming ~51% less computation time. Our findings highlight the effectiveness of exploiting internal representation decoupling in LMs for efficient and practical hallucination detection.

幻觉检测语言模型单次推理表示解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。