用中间层激活融合检测大模型幻觉，单次前向传播即可完成。
Prediction of Prediction (PoP): Inter-Layer Activation Fusion for Single-Pass Hallucination Detection in Large Language Models
- 通过融合不同层的隐藏状态，捕捉生成过程中的不确定性。
- 在TruthfulQA上实现75.5%的AUROC，无需额外推理步骤。
- 适合对实时性与准确性要求高的高风险应用场景。
自回归大语言模型常以高置信度生成事实错误内容，限制其在高风险场景的应用。现有输出阶段的不确定性度量在模型对错误陈述过度自信时失效，而多样本验证则带来显著内存和延迟开销。本文评估了生成过程中内部隐藏状态演变是否可作为事实错误的信号，无需额外解码。提出预测预测（PoP）机制，通过单次前向传播中跨深度融合中间隐藏表示，捕捉层间过渡不确定性。在使用自回归Transformer骨干网络的TruthfulQA基准上，PoP在事实正确性分类任务中达到75.5%的受试者工作特征曲线下面积（AUROC）。该机制运行于基础前向传播内，增加的运行时延迟不足1.2%，且无需额外生成步骤。所有数值结果均来自作者验证的实验实现，并受限于下述评估范围。
原文摘要 · Abstract (English)
Autoregressive large language models (LLMs) routinely generate factually incorrect outputs with high decoding confidence, limiting their deployment in high-stakes workflows. Existing output-stage uncertainty metrics can fail when models are overconfident on false assertions, while multi-sample verification pipelines introduce substantial memory and latency overhead. This work evaluates whether internal hidden-state transition dynamics during generation can signal factual errors without auxiliary decoding calls. We introduce Prediction of Prediction (PoP), a mechanism that captures layer-transition uncertainty by fusing intermediate hidden representations across depth during a single forward pass. Evaluated on the TruthfulQA benchmark using autoregressive transformer backbones, PoP achieves an area under the receiver operating characteristic curve (AUROC) of 75.5% for factual-correctness classification. The mechanism operates within the base forward pass, adding less than 1.2% runtime latency and requiring zero additional generation passes. The numerical results are reported from the author-verified experimental implementation and are bounded by the evaluation scope described below.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。