用大模型提升医疗问诊摘要精准度,聚焦核心问题防幻觉
FocusMed: A Large Language Model-based Framework for Enhancing Medical Question Summarization with Focus Identification
- 通过提示工程引导模型提取原始问题的核心焦点
- 在双数据集上实现领先性能,焦点识别准确率显著提升
- 适合医疗AI研发、临床辅助系统开发人员参考
随着在线医疗平台快速发展,患者健康咨询(CHQs)因冗余信息和非专业术语导致诊断效率低下。医学问题摘要(MQS)任务旨在将患者提问转化为医生常问问题(FAQ),但现有方法仍存在焦点识别不准和模型幻觉等问题。本文探索大语言模型(LLMs)在MQS中的潜力,发现直接微调易产生焦点偏差并生成不忠实内容。为此,提出基于核心焦点引导的优化框架:首先设计提示模板,引导模型忠实提取问题核心焦点;其次结合原始CHQ-FAQ对构建微调数据集,增强焦点识别能力;最后提出多维度质量评估与选择机制,从多个角度提升摘要质量。在两个主流MQS数据集上,使用三个评估指标进行综合实验,所提框架在所有指标上均达到当前最优表现,显著提升焦点识别能力并有效缓解幻觉问题。源代码已开源:https://github.com/DUT-LiuChao/FocusMed。
原文摘要 · Abstract (English)
With the rapid development of online medical platforms, consumer health questions (CHQs) are inefficient in diagnosis due to redundant information and frequent non-professional terms. The medical question summary (MQS) task aims to transform CHQs into streamlined doctors' frequently asked questions (FAQs), but existing methods still face challenges such as poor identification of question focus and model hallucination. This paper explores the potential of large language models (LLMs) in the MQS task and finds that direct fine-tuning is prone to focus identification bias and generates unfaithful content. To this end, we propose an optimization framework based on core focus guidance. First, a prompt template is designed to drive the LLMs to extract the core focus from the CHQs that is faithful to the original text. Then, a fine-tuning dataset is constructed in combination with the original CHQ-FAQ pairs to improve the ability to identify the focus of the question. Finally, a multi-dimensional quality evaluation and selection mechanism is proposed to comprehensively improve the quality of the summary from multiple dimensions. We conduct comprehensive experiments on two widely-adopted MQS datasets using three established evaluation metrics. The proposed framework achieves state-of-the-art performance across all measures, demonstrating a significant boost in the model's ability to identify critical focus of questions and a notable mitigation of hallucinations. The source codes are freely available at https://github.com/DUT-LiuChao/FocusMed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。