用细粒度事实核查+医疗领域适配，降低大模型医疗幻觉。
Mitigating hallucinations in healthcare LLMs with granular fact-checking and domain-specific adaptation
- 独立于大模型的事实核查模块，结合数值验证与逻辑分析。
- 事实核查精度达89.04%，召回率82.34%，幻觉显著减少。
- 专为医疗文本优化，适合临床决策支持系统使用。
在医疗领域，大型语言模型（LLM）生成内容的可靠性至关重要，尤其涉及诊疗决策与患者安全时。然而，由于模型幻觉风险，输出常不可靠。为此，我们提出一个独立于任何LLM的事实核查模块，以及一个针对医疗领域的摘要模型，以降低幻觉率。该模型基于低秩适应（LoRA）在MIMIC-III数据集上微调，并搭配事实核查模块，通过自然语言处理中的离散逻辑对电子健康记录（EHRs）进行数值验证与细粒度逻辑检查。评估中，从104份摘要中提取3786个命题作为事实基准，事实核查模块达到0.8904的精确率、0.8234的召回率和0.8556的F1分数。同时，该模型生成摘要的ROUGE-1得分为0.5797，BERTScore为0.9120，质量良好。
原文摘要 · Abstract (English)
In healthcare, it is essential for any Large Language Model (LLM)-generated output to be reliable and accurate, particularly in cases involving decision-making and patient safety. However, the outputs are often unreliable in such critical areas due to the risk of hallucinated outputs from the LLMs. To address this issue, we propose a fact-checking module that operates independently of any LLM, along with a domain-specific summarization model designed to minimize hallucination rates. Our model is fine-tuned using Low-Rank Adaptation (LoRA) on the MIMIC-III dataset and is paired with the fact-checking module, which uses numerical tests for correctness and logical checks at a granular level through discrete logic in natural language processing (NLP) to validate facts against electronic health records (EHRs). We trained the LLM on the full MIMIC-III dataset. For evaluation of the fact-checking module, we sampled 104 summaries, extracted them into 3786 propositions, and used these as facts. The fact-checking module achieves a precision of 0.8904, a recall of 0.8234, and an F1-score of 0.8556. Additionally, the LLM summary achieves a ROUGE-1 score of 0.5797 and a BERTScore of 0.9120 for summary quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。