让大模型自己学会识别幻觉,训练时用弱监督信号提炼检测能力。
Weakly Supervised Distillation of Hallucination Signals into Transformer Representations
- 用三种无人工标注的接地信号构建数据集,弱化对外部验证的依赖。
- 基于隐藏状态训练探测器,跨层注意力模型在测试集上达到最高AUC/F1。
- 推理时仅靠内部激活就能检测幻觉,计算开销极小,适合实际部署。
现有大语言模型幻觉检测方法依赖推理时的外部验证(如标准答案、检索系统或辅助判别模型)。本文探讨能否将此类外部监督在训练阶段蒸馏进模型自身的表征中,实现仅凭内部激活进行幻觉检测。提出一种弱监督框架,融合子串匹配、句子嵌入相似度及大模型判别结果作为三类互补的接地信号,无需人工标注即可对生成回答打上「接地」或「幻觉」标签。基于SQuAD v2构建15000样本数据集(10500训练/开发样本,5000测试样本),每例包含LLaMA-2-7B生成的答案及其全层隐藏状态与结构化标签。在此基础上,直接在隐藏状态上训练五种探测器:ProbeMLP (M0)、LayerWiseMLP (M1)、CrossLayerTransformer (M2)、HierarchicalTransformer (M3) 和 CrossLayerAttentionTransformerV2 (M4),仅将外部接地信号作为训练监督。实验支持核心假设:幻觉检测信号可被蒸馏至变换器表征中,使推理时无需外部验证。基于变换器的探测器表现最优,其中M2在五折平均AUC/F1上最佳,M3在单折验证和独立测试集上均最优。同时评估推理效率:探测延迟为0.15–5.62毫秒(批量)和1.55–6.66毫秒(单样本),端到端生成+探测吞吐量约0.231查询/秒,表明实际开销可忽略。
原文摘要 · Abstract (English)
Existing hallucination detection methods for large language models (LLMs) rely on external verification at inference time, requiring gold answers, retrieval systems, or auxiliary judge models. We ask whether this external supervision can instead be distilled into the model's own representations during training, enabling hallucination detection from internal activations alone at inference time. We introduce a weak supervision framework that combines three complementary grounding signals: substring matching, sentence embedding similarity, and an LLM as a judge verdict to label generated responses as grounded or hallucinated without human annotation. Using this framework, we construct a 15000-sample dataset from SQuAD v2 (10500 train/development samples and a separate 5000-sample test set), where each example pairs a LLaMA-2-7B generated answer with its full per-layer hidden states and structured hallucination labels. We then train five probing classifiers: ProbeMLP (M0), LayerWiseMLP (M1), CrossLayerTransformer (M2), HierarchicalTransformer (M3), and CrossLayerAttentionTransformerV2 (M4), directly on these hidden states, treating external grounding signals as training-time supervision only. Our central hypothesis is that hallucination detection signals can be distilled into transformer representations, enabling internal detection without any external verification at inference time. Results support this hypothesis. Transformer-based probes achieve the strongest discrimination, with M2 performing best on 5-fold average AUC/F1, and M3 performing best on both single-fold validation and held-out test evaluation. We also benchmark inference efficiency: probe latency ranges from 0.15 to 5.62 ms (batched) and 1.55 to 6.66 ms (single sample), while end-to-end generation plus probe throughput remains approximately 0.231 queries per second, indicating negligible practical overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。