通过扰动模型参数提升幻觉检测能力，无需训练即可显著增强效果。
Enhancing Hallucination Detection through Noise Injection
- 采样时扰动模型参数或隐藏层激活值，捕捉贝叶斯意义下的不确定性
- 在多个数据集和模型上，检测准确率明显优于标准采样方法
- 方法简单且无需训练，适合实际部署中的实时幻觉检测
大型语言模型（LLMs）容易生成看似合理但错误的回答，即幻觉。有效检测幻觉对LLM的安全部署至关重要。近期研究发现幻觉与模型不确定性相关，可通过多次采样后答案分布的离散度来检测。虽然从模型定义的词元分布中采样是自然方式，但本文认为其对幻觉检测效果不佳。我们提出一种简单、无需训练的方法：在采样过程中扰动模型参数或等效的隐藏单元激活值，以更充分地反映贝叶斯不确定性。实验表明，该方法在多种数据集、模型架构和不确定性度量下，均显著提升推理阶段的幻觉检测性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are prone to generating plausible yet incorrect responses, known as hallucinations. Effectively detecting hallucinations is therefore crucial for the safe deployment of LLMs. Recent research has linked hallucinations to model uncertainty, suggesting that hallucinations can be detected by measuring dispersion over answer distributions obtained from multiple samples drawn from a model. While drawing from the distribution over tokens defined by the model is a natural way to obtain samples, in this work, we argue that it is suboptimal for the purpose of detecting hallucinations. We show that detection can be improved significantly by taking into account model uncertainty in the Bayesian sense. To this end, we propose a very simple, training-free approach based on perturbing an appropriate subset of model parameters, or equivalently hidden unit activations, during sampling. We demonstrate that our approach significantly improves inference-time hallucination detection over standard sampling across diverse datasets, model architectures, and uncertainty metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。