arXiv:2409.17504cs.LGcs.CL2024-09NeurIPS被引 123

利用未标注的LLM生成内容自动检测幻觉,无需人工标注。

HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection

论文配图:HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection
图 1 · 摘自论文原文
  • 通过自动化成员估计得分区分真实与虚假生成内容。
  • 在无额外标注数据下实现优于现有方法的检测性能。
  • 适合需要低成本部署幻觉检测系统的实际应用。

大型语言模型(LLMs)应用的激增引发了对其生成误导性或虚构信息(即幻觉)的担忧。因此,检测幻觉对于维护LLM生成内容的信任至关重要。学习可信度分类器的主要挑战在于缺乏大量标注的真实与幻觉数据。为解决该问题,我们提出HaloScope,一种新颖的学习框架,利用开放世界中部署LLM时产生的未标注生成内容进行幻觉检测。这些未标注数据自然产生,包含真实与幻觉信息。为挖掘此类数据,我们提出一种自动化成员估计得分,可在未标注混合数据中区分真实与虚假生成内容,从而支持训练二元可信度分类器。重要的是,该框架无需额外数据收集或人工标注,具备强灵活性与实用性。大量实验表明,HaloScope能显著超越竞争方法,实现更优的幻觉检测性能。代码已公开于 https://github.com/deeplearningwisc/haloscope。

原文摘要 · Abstract (English)

The surge in applications of large language models (LLMs) has prompted concerns about the generation of misleading or fabricated information, known as hallucinations. Therefore, detecting hallucinations has become critical to maintaining trust in LLM-generated content. A primary challenge in learning a truthfulness classifier is the lack of a large amount of labeled truthful and hallucinated data. To address the challenge, we introduce HaloScope, a novel learning framework that leverages the unlabeled LLM generations in the wild for hallucination detection. Such unlabeled data arises freely upon deploying LLMs in the open world, and consists of both truthful and hallucinated information. To harness the unlabeled data, we present an automated membership estimation score for distinguishing between truthful and untruthful generations within unlabeled mixture data, thereby enabling the training of a binary truthfulness classifier on top. Importantly, our framework does not require extra data collection and human annotations, offering strong flexibility and practicality for real-world applications. Extensive experiments show that HaloScope can achieve superior hallucination detection performance, outperforming the competitive rivals by a significant margin. Code is available at https://github.com/deeplearningwisc/haloscope.

幻觉检测无监督学习LLM安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。