arXiv:2502.15844cs.CLcs.LG2025-02中稿 · the ACM Joint Euro…被引 50

不依赖外部数据,用提示词变异检测大模型幻觉。

Hallucination Detection in Large Language Models with Metamorphic Relations

  • 通过设计提示词变异和元关系验证,自洽检测幻觉。
  • 在四个模型上平均F1提升112.2%,最高超36.8%。
  • 适合需高准确率且无法访问模型内部的场景。

大语言模型在生成回答时容易出现事实性错误,即幻觉,这对高准确率要求的应用构成挑战。现有检测方法多依赖外部资源,存在可用性低、覆盖不全、隐私风险、延迟高、可靠性差和可扩展性差等问题;也有方法依赖输出概率,但对闭源模型如GPT不可行。本文提出MetaQA,一种无需外部资源的自包含幻觉检测方法,利用元关系与提示词变异。其核心假设是:若模型回答存在幻觉,则设计的元关系将被破坏。我们在多个数据集上,对比MetaQA与最先进的零资源检测方法SelfCheckGPT,涵盖两个开源和两个闭源模型。结果表明,MetaQA在精度、召回率和F1分数上均优于SelfCheckGPT,四款模型中,精度提升0.041–0.113,召回率提升0.143–0.430,F1提升0.154–0.368。以Mistral-7B为例,MetaQA平均F1达0.435,优于SelfCheckGPT的0.205,提升率达112.2%。该方法在各类问题类别中均表现更优。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are prone to hallucinations, e.g., factually incorrect information, in their responses. These hallucinations present challenges for LLM-based applications that demand high factual accuracy. Existing hallucination detection methods primarily depend on external resources, which can suffer from issues such as low availability, incomplete coverage, privacy concerns, high latency, low reliability, and poor scalability. There are also methods depending on output probabilities, which are often inaccessible for closed-source LLMs like GPT models. This paper presents MetaQA, a self-contained hallucination detection approach that leverages metamorphic relation and prompt mutation. Unlike existing methods, MetaQA operates without any external resources and is compatible with both open-source and closed-source LLMs. MetaQA is based on the hypothesis that if an LLM's response is a hallucination, the designed metamorphic relations will be violated. We compare MetaQA with the state-of-the-art zero-resource hallucination detection method, SelfCheckGPT, across multiple datasets, and on two open-source and two closed-source LLMs. Our results reveal that MetaQA outperforms SelfCheckGPT in terms of precision, recall, and f1 score. For the four LLMs we study, MetaQA outperforms SelfCheckGPT with a superiority margin ranging from 0.041 - 0.113 (for precision), 0.143 - 0.430 (for recall), and 0.154 - 0.368 (for F1-score). For instance, with Mistral-7B, MetaQA achieves an average F1-score of 0.435, compared to SelfCheckGPT's F1-score of 0.205, representing an improvement rate of 112.2%. MetaQA also demonstrates superiority across all different categories of questions.

幻觉检测大模型自洽验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。