用检索文档检测大模型幻觉，跨语言表现优异。
REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models
- 通过检索文档直接判断输出是否幻觉
- 在九种语言上实现更高精确率和召回率
- 适合需要可靠生成结果的多语言应用
大语言模型（LLM）输出中的幻觉严重影响其在问答等知识密集型任务中的可靠性。为此，我们提出REFIND（Retrieval-augmented Factuality hallucINation Detection），一种通过直接利用检索到的文档来检测LLM输出中幻觉片段的新框架。作为REFIND的一部分，我们引入了上下文敏感度比（CSR），一种量化LLM输出对检索证据敏感性的新指标。该方法使REFIND能够高效准确地检测幻觉，优于现有方法。评估结果显示，REFIND在包括低资源语言在内的九种语言中均表现出鲁棒性，显著超越基线模型，在识别幻觉片段方面取得了更优的交并比（IoU）得分。本工作验证了量化上下文敏感性在幻觉检测中的有效性，为跨多种语言的更可靠、可信的LLM应用铺平道路。代码已开源：https://github.com/oneonlee/REFIND。
原文摘要 · Abstract (English)
Hallucinations in large language model (LLM) outputs severely limit their reliability in knowledge-intensive tasks such as question answering. To address this challenge, we introduce REFIND (Retrieval-augmented Factuality hallucINation Detection), a novel framework that detects hallucinated spans within LLM outputs by directly leveraging retrieved documents. As part of the REFIND, we propose the Context Sensitivity Ratio (CSR), a novel metric that quantifies the sensitivity of LLM outputs to retrieved evidence. This innovative approach enables REFIND to efficiently and accurately detect hallucinations, setting it apart from existing methods. In the evaluation, REFIND demonstrated robustness across nine languages, including low-resource settings, and significantly outperformed baseline models, achieving superior IoU scores in identifying hallucinated spans. This work highlights the effectiveness of quantifying context sensitivity for hallucination detection, thereby paving the way for more reliable and trustworthy LLM applications across diverse languages. Our code is available at https://github.com/oneonlee/REFIND.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。