提出实时检测长文本幻觉实体的方法,适用于大模型实际部署。
Real-Time Detection of Hallucinated Entities in Long-Form Generation
- 聚焦实体级幻觉,通过网页搜索标注生成内容中的虚构信息
- 在Llama-3.3-70B上实现AUC 0.90,优于基线方法(0.71)
- 仅用线性探针即可有效检测,且可跨模型迁移使用
大型语言模型已广泛用于医疗咨询、法律建议等高风险场景,幻觉可能造成严重后果。现有检测方法或仅限短文本查询,或需昂贵的外部验证,难以实用。本文提出一种低成本、可扩展的实时方法,用于识别长文本生成中的幻觉实体(如虚构姓名、日期、引用),并成功应用于700亿参数模型。该方法聚焦实体级幻觉,天然适配词级别标签,支持流式检测。我们设计了一种基于网络搜索的标注方法,为模型输出打上真实性的标签。基于此数据集,仅用简单高效的线性探针即可训练出有效的幻觉分类器。在四个模型家族上评估,该方法在长文本生成中持续优于基线,包括更复杂的语义熵方法(如对Llama-3.3-70B,AUC达0.90,基线为0.71),并在短文本问答中同样表现更好。尽管标注成本较高,但发现一个模型的标注数据可用于训练其他模型的分类器,因此公开发布数据集以促进复用。结果表明,该方法具备良好的泛化能力,不仅检测实体幻觉,还能有效识别数学推理中的错误答案。整体上,本工作为可扩展的真实世界幻觉检测提供了新路径。
原文摘要 · Abstract (English)
Large language models are now routinely used in high-stakes applications where hallucinations can cause serious harm, such as medical consultations or legal advice. Existing hallucination detection methods, however, are impractical for real-world use, as they are either limited to short factual queries or require costly external verification. We present a cheap, scalable method for real-time identification of hallucinated tokens in long-form generations, and scale it effectively to 70B parameter models. Our approach targets entity-level hallucinations-e.g., fabricated names, dates, citations-rather than claim-level, thereby naturally mapping to token-level labels and enabling streaming detection. We develop an annotation methodology that leverages web search to annotate model responses with grounded labels indicating which tokens correspond to fabricated entities. This dataset enables us to train effective hallucination classifiers with simple and efficient methods such as linear probes. Evaluating across four model families, our classifiers consistently outperform baselines on long-form responses, including more expensive methods such as semantic entropy (e.g., AUC 0.90 vs 0.71 for Llama-3.3-70B), and are also an improvement in short-form question-answering settings. Despite being trained only to detect hallucinated entities, our probes effectively detect incorrect answers in mathematical reasoning tasks, indicating generalization beyond entities. While our annotation methodology is expensive, we find that annotated responses from one model can be used to train effective classifiers on other models; accordingly, we publicly release our datasets to facilitate reuse. Overall, our work suggests a promising new approach for scalable, real-world hallucination detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。