区分大模型幻觉是无知还是错误，有助于更精准地修复问题。
Distinguishing Ignorance from Error in LLM Hallucinations
- 将幻觉分为无知识型和有知识但答错型两类。
- 发现不同模型在不同样本上犯错，且错误类型普遍存在。
- 建议为每种模型定制幻觉数据集以提升检测效果。
大型语言模型容易产生事实性错误输出，即幻觉。本文提出区分两类幻觉：一类是模型参数中缺乏正确答案（称为HK-），另一类是模型虽有知识却仍回答错误（称为HK+）。研究发现HK+幻觉普遍存在，跨模型和数据集均可见。进一步证明，区分两类幻觉有助于更有效缓解幻觉问题。重要的是，不同模型在不同例子上出现幻觉，这促使构建针对特定模型的幻觉数据集以训练检测器。整体研究强调对幻觉类型进行分类的重要性，并提供了更有效的应对方法。代码已公开于 https://github.com/technion-cs-nlp/hallucination-mitigation。
原文摘要 · Abstract (English)
Large language models (LLMs) are susceptible to hallucinations -- factually incorrect outputs -- leading to a large body of work on detecting and mitigating such cases. We argue that it is important to distinguish between two types of hallucinations: ones where the model does not hold the correct answer in its parameters, which we term HK-, and ones where the model answers incorrectly despite having the required knowledge, termed HK+. We first find that HK+ hallucinations are prevalent and occur across models and datasets. Then, we demonstrate that distinguishing between these two cases is beneficial for mitigating hallucinations. Importantly, we show that different models hallucinate on different examples, which motivates constructing model-specific hallucination datasets for training detectors. Overall, our findings draw attention to classifying types of hallucinations and provide means to handle them more effectively. The code is available at https://github.com/technion-cs-nlp/hallucination-mitigation .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。