首个专用于检测医疗大模型幻觉的基准，揭示现有模型在识别错误信息上表现不佳。
MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models
- 构建1万条医学问答对，通过可控流程生成幻觉答案以评估模型可靠性。
- 顶尖模型在难检测幻觉上F1仅0.625，表明当前技术仍存在严重缺陷。
- 引入领域知识与'不确定'选项可使准确率提升38%，适合医疗AI安全研究者。
大型语言模型（LLMs）在医学问答中的应用日益广泛，但其可靠性面临幻觉挑战——模型生成看似合理却事实错误的内容，可能危及患者安全与临床决策。为此，我们提出MedHallu，首个专为医疗幻觉检测设计的基准。该基准包含从PubMedQA中提取的10,000个高质量问答对，并通过受控流程系统生成幻觉答案。实验显示，包括GPT-4o、Llama-3.1和医学微调版UltraMedical在内的先进模型在二分类幻觉检测任务中表现有限，最佳模型在“难”类别上的F1仅为0.625。通过双向蕴含聚类分析发现，难以检测的幻觉在语义上更接近真实答案。进一步实验表明，引入领域知识并设置“不确定”作为答案选项，可使精确率和F1分数相对基线提升最高达38%。
原文摘要 · Abstract (English)
Advancements in Large Language Models (LLMs) and their increasing use in medical question-answering necessitate rigorous evaluation of their reliability. A critical challenge lies in hallucination, where models generate plausible yet factually incorrect outputs. In the medical domain, this poses serious risks to patient safety and clinical decision-making. To address this, we introduce MedHallu, the first benchmark specifically designed for medical hallucination detection. MedHallu comprises 10,000 high-quality question-answer pairs derived from PubMedQA, with hallucinated answers systematically generated through a controlled pipeline. Our experiments show that state-of-the-art LLMs, including GPT-4o, Llama-3.1, and the medically fine-tuned UltraMedical, struggle with this binary hallucination detection task, with the best model achieving an F1 score as low as 0.625 for detecting "hard" category hallucinations. Using bidirectional entailment clustering, we show that harder-to-detect hallucinations are semantically closer to ground truth. Through experiments, we also show incorporating domain-specific knowledge and introducing a "not sure" category as one of the answer categories improves the precision and F1 scores by up to 38% relative to baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。