用简单探针识别大模型答案来自记忆还是上下文，提升幻觉检测精度。
Probing for Knowledge Attribution in Large Language Models
- 通过自监督数据生成管道自动标注知识来源，训练线性探针识别输出来源。
- 在多个模型上达到0.96的宏平均F1，跨任务零样本迁移性能优异。
- 发现错误归因会使错误率上升70%，适合关注幻觉机制的研究者阅读。
大型语言模型的幻觉分为两类：忠实性违规（误用上下文）和事实性违规（内部知识错误）。有效缓解依赖于明确每个回答的知识来源。本文研究贡献归因，即判断输出主导知识来源的分类问题，证明仅用隐藏表示训练的线性探针即可可靠识别。提出AttriWiki自监督流水线，通过提示模型回忆被隐藏的实体或从上下文读取，无需依赖知识冲突即可自动生成带标签训练数据。在Llama-3.1-8B、Mistral-7B和Qwen-7B上，探针达到最高0.96的宏平均F1；在SQuAD和WebQuestions上转移性能为0.94–0.99；在Tighidet等（2024）基准上实现零样本泛化，优于原探针且无需重训。此外，归因错误会使错误率升高至70%，但正确归因仍无法保证答案正确，凸显需构建更全面的检测框架。
原文摘要 · Abstract (English)
Large language model (LLM) hallucinations, meaning fluent but factually incorrect generations, fall into two types: faithfulness violations, where the model misuses provided context, and factuality violations, where answers reflect errors in internal knowledge. Proper mitigation depends on knowing which source drives each answer. We study contributive attribution, i.e. the classification of the dominant knowledge source behind each output, and show that a simple linear probe trained on hidden representations can reliably identify it. We introduce AttriWiki, a self-supervised pipeline that automatically generates labelled training data by prompting models to recall withheld entities from memory or read them from context without relying on knowledge conflicts. Probes trained on AttriWiki achieve up to 0.96 Macro-$F_1$ on Llama-3.1-8B, Mistral-7B, and Qwen-7B, transfer to SQuAD and WebQuestions with 0.94-0.99 Macro-$F_1$, and generalise zero-shot to Tighidet et al. (2024)'s benchmark, outperforming their probe on conflicting settings without retraining. Furthermore, attribution mismatches raise error rates by up to 70%, though correct attribution does not guarantee correct answers, pointing to the need for broader detection frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。