提前检测大模型幻觉风险，提升回答可靠性。
Pre-Generation Hallucination Detection in Large Language Models via Soft-Target Attention Probing
- 用采样输出误差率构建软标签，更精准估计幻觉概率。
- 在三个问答数据集上，检测准确率显著优于传统方法。
- 适合需要高可靠性的问答系统和生成决策场景。
在生成前检测幻觉风险，可实现拒绝回答、检索增强和路由决策，无需承担解码成本。尽管已有研究证明可通过模型内部表示估算风险，但现有方法仅将其视为单一输出的二分类问题。本文将其重新定义为风险估计问题，提出基于随机采样输出经验误差率的软目标监督，我们证明该估计量是在模型采样分布下唯一无偏且方差最小的每提示误差概率估计器。进一步将注意力探针适配至生成前场景，使探测器能选择性聚合与幻觉相关的提示表征。在三个问答基准和五个模型上，注意力探针在短答案任务中优于线性探针；使用软目标监督进一步且持续提升了检测质量。
原文摘要 · Abstract (English)
Detecting hallucination risk before generation enables abstention, retrieval augmentation, and routing decisions without incurring the cost of decoding. While prior work has shown that such risk can be estimated from a model's internal representations, existing approaches treat this as binary classification over a single decoded output. We instead formulate it as a risk-estimation problem. Under this formulation, we introduce soft-target supervision based on the empirical answer error rate over stochastically sampled outputs - an estimator we prove to be the unique unbiased minimum-variance estimator of the model's per-prompt error probability under its sampling distribution. We further adapt attention probing to the pre-generation setting, enabling the detector to selectively aggregate hallucination-relevant prompt representations. Across three question-answering benchmarks and five models, attention probing outperforms linear probing on short-answer tasks. Replacing binary labels with soft-target supervision further and consistently improves detection quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。