arXiv:2509.11536cs.CLcs.AI2025-09被引 6

通过分解模型推理空间,实现更精准的幻觉检测。

HARP: Hallucination Detection via Reasoning Subspace Projection

  • 将大模型隐藏状态分解为语义与推理子空间,分离信息维度。
  • 在TriviaQA数据集上达到92.8%的AUROC,比前最优方法高7.5%。
  • 特征维度压缩至原大小5%,有效降噪并提升鲁棒性,适合部署场景。

大型语言模型(LLMs)中的幻觉问题严重阻碍其在关键决策中的可靠应用。尽管现有检测方法已提升准确性,但仍难以区分语义与推理信息,且缺乏鲁棒性。为此,我们提出HARP(基于推理子空间投影的幻觉检测),首次证明大模型的隐藏状态空间可分解为语义子空间与推理子空间的直和。其中,语义子空间编码语言表达,推理子空间捕捉内部推理过程。我们进一步发现,反嵌入层可解耦这两个子空间,通过对其参数进行奇异值分解(SVD),获得构成两子空间的基向量。随后,将隐藏状态投影至推理子空间的基向量上,所得投影作为幻觉检测的输入特征。该方法将特征维度降至原始的约5%,滤除大部分噪声,显著提升鲁棒性。多数据集实验表明,HARP达到当前最佳性能;尤其在TriviaQA上,其AUROC达92.8%,超越先前最优方法7.5个百分点。

原文摘要 · Abstract (English)

Hallucinations in Large Language Models (LLMs) pose a major barrier to their reliable use in critical decision-making. Although existing hallucination detection methods have improved accuracy, they still struggle with disentangling semantic and reasoning information and maintaining robustness. To address these challenges, we propose HARP (Hallucination detection via reasoning subspace projection), a novel hallucination detection framework. HARP establishes that the hidden state space of LLMs can be decomposed into a direct sum of a semantic subspace and a reasoning subspace, where the former encodes linguistic expression and the latter captures internal reasoning processes. Moreover, we demonstrate that the Unembedding layer can disentangle these subspaces, and by applying Singular Value Decomposition (SVD) to its parameters, the basis vectors spanning the semantic and reasoning subspaces are obtained. Finally, HARP projects hidden states onto the basis vectors of the reasoning subspace, and the resulting projections are then used as input features for hallucination detection in LLMs. By using these projections, HARP reduces the dimension of the feature to approximately 5% of the original, filters out most noise, and achieves enhanced robustness. Experiments across multiple datasets show that HARP achieves state-of-the-art hallucination detection performance; in particular, it achieves an AUROC of 92.8% on TriviaQA, outperforming the previous best method by 7.5%.

幻觉检测子空间分析大模型鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。