Medico通过多源证据融合，自动检测并修正大模型幻觉内容。
Medico: Towards Hallucination Detection and Correction with Multi-source Evidence Fusion
- 融合多源证据，自动判断生成内容是否含事实错误。
- 在检索、检测和修正任务上分别达到0.964、0.951、0.979的优异表现。
- 适合医疗、法律等对事实准确性要求高的领域使用。
大语言模型中普遍存在幻觉问题，即生成内容看似合理但事实错误，严重制约其广泛应用。已有研究发现，大模型倾向于自信地陈述不存在的事实，而非回答“我不知道”。因此，有必要引入外部知识实现幻觉的检测与纠正。由于人工检测纠错成本高，亟需自动化端到端的解决方案。为此，我们提出Medico——一种基于多源证据融合的幻觉检测与修正框架。该框架整合多源证据，识别生成内容中的事实错误，提供判断依据，并迭代修正幻觉内容。在证据检索（HR@5=0.964，MRR@5=0.908）、幻觉检测（F1=0.927–0.951）和幻觉修正（批准率=0.973–0.979）任务上的实验结果表明，Medico具有显著潜力。视频演示可访问：https://youtu.be/RtsO6CSesBI。
原文摘要 · Abstract (English)
As we all know, hallucinations prevail in Large Language Models (LLMs), where the generated content is coherent but factually incorrect, which inflicts a heavy blow on the widespread application of LLMs. Previous studies have shown that LLMs could confidently state non-existent facts rather than answering ``I don't know''. Therefore, it is necessary to resort to external knowledge to detect and correct the hallucinated content. Since manual detection and correction of factual errors is labor-intensive, developing an automatic end-to-end hallucination-checking approach is indeed a needful thing. To this end, we present Medico, a Multi-source evidence fusion enhanced hallucination detection and correction framework. It fuses diverse evidence from multiple sources, detects whether the generated content contains factual errors, provides the rationale behind the judgment, and iteratively revises the hallucinated content. Experimental results on evidence retrieval (0.964 HR@5, 0.908 MRR@5), hallucination detection (0.927-0.951 F1), and hallucination correction (0.973-0.979 approval rate) manifest the great potential of Medico. A video demo of Medico can be found at https://youtu.be/RtsO6CSesBI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。