arXiv:2509.21357cs.CLcs.AI2025-09

通过差异特征学习定位幻觉信号,实现高效精准检测。

A Novel Differential Feature Learning for Effective Hallucination Detection and Classification

  • 设计双模型架构,用差分特征学习捕捉幻觉与真实内容差异。
  • 发现幻觉信号集中在极少数稀疏特征上,仅用1%维度即可保持高精度。
  • 揭示浅层多样、深层聚焦的层级“漏斗模式”,适合资源受限场景。

大语言模型幻觉是因训练数据分布偏差导致输出偏离事实的关键挑战。尽管已有研究指出特定隐藏层在幻觉与真实内容间存在差异,但幻觉信号在层内的精确位置仍不明确,制约了高效检测方法的发展。本文提出一种双模型架构,集成投影融合(PF)模块以自适应加权跨层特征,并引入差异特征学习(DFL)机制,通过计算并行编码器对相同输入学习互补表征后的差异,识别判别性特征。在HaluEval的问答、对话和摘要数据集上系统实验表明,幻觉信号集中于高度稀疏的特征子集,问答与对话任务准确率显著提升。值得注意的是,分析揭示了一种层级“漏斗模式”:浅层特征多样性高,深层使用集中,仅保留1%特征维度即可维持检测性能且降损极小。这些发现表明幻觉信号比以往认为的更为集中,为计算高效的检测系统提供了路径,有望在保持精度的同时降低推理成本。

原文摘要 · Abstract (English)

Large language model hallucination represents a critical challenge where outputs deviate from factual accuracy due to distributional biases in training data. While recent investigations establish that specific hidden layers exhibit differences between hallucinatory and factual content, the precise localization of hallucination signals within layers remains unclear, limiting the development of efficient detection methods. We propose a dual-model architecture integrating a Projected Fusion (PF) block for adaptive inter-layer feature weighting and a Differential Feature Learning (DFL) mechanism that identifies discriminative features by computing differences between parallel encoders learning complementary representations from identical inputs. Through systematic experiments across HaluEval's question answering, dialogue, and summarization datasets, we demonstrate that hallucination signals concentrate in highly sparse feature subsets, achieving significant accuracy improvements on question answering and dialogue tasks. Notably, our analysis reveals a hierarchical "funnel pattern" where shallow layers exhibit high feature diversity while deep layers demonstrate concentrated usage, enabling detection performance to be maintained with minimal degradation using only 1\% of feature dimensions. These findings suggest that hallucination signals are more concentrated than previously assumed, offering a pathway toward computationally efficient detection systems that could reduce inference costs while maintaining accuracy.

幻觉检测特征学习高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。