arXiv:2506.00448cs.CL2025-06被引 2

针对医疗对话摘要中的幻觉问题,提出可解释的检测方法

Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization

  • 通过系统性删减事实生成可控幻觉数据集
  • 通用幻觉检测器在临床场景表现不佳,新方法更有效
  • 适合医疗AI安全、临床系统开发者使用

大语言模型在患者-医生对话摘要中产生幻觉,严重威胁患者安全和临床决策。然而该现象在临床领域仍研究不足,通用幻觉检测器的适用性存疑。由于幻觉稀少且随机,研究难度大。本文构建两个数据集:基于事实移除的留N-out可控幻觉数据集,以及基于LLM生成的自然幻觉数据集。实验表明,通用检测器难以识别临床幻觉,且在可控幻觉上的表现无法预测对自然幻觉的检测效果。为此,我们提出基于事实的幻觉计数方法,提供可解释性。特别地,利用可控幻觉数据训练的LLM检测器,在真实临床幻觉上表现出良好泛化能力。本研究提供了专家标注的数据集与专用评估指标,推动可信医疗摘要系统发展。

原文摘要 · Abstract (English)

Hallucinations in large language models (LLMs) during summarization of patient-clinician dialogues pose significant risks to patient care and clinical decision-making. However, the phenomenon remains understudied in the clinical domain, with uncertainty surrounding the applicability of general-domain hallucination detectors. The rarity and randomness of hallucinations further complicate their investigation. In this paper, we conduct an evaluation of hallucination detection methods in the medical domain, and construct two datasets for the purpose: A fact-controlled Leave-N-out dataset -- generated by systematically removing facts from source dialogues to induce hallucinated content in summaries; and a natural hallucination dataset -- arising organically during LLM-based medical summarization. We show that general-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations. We then develop fact-based approaches that count hallucinations, offering explainability not available with existing methods. Notably, our LLM-based detectors, which we developed using fact-controlled hallucinations, generalize well to detecting real-world clinical hallucinations. This research contributes a suite of specialized metrics supported by expert-annotated datasets to advance faithful clinical summarization systems.

幻觉检测医疗AI可解释性大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。