arXiv:2506.11137cs.CL2025-06被引 6

用大模型从病历中自动提取药物并判断停药,效果接近人工标注。

Scalable Medication Extraction and Discontinuation Identification from Electronic Health Records Using Large Language Models

  • 用大模型直接读取未标注的电子病历,自动识别用药和停药情况。
  • GPT-4o在无标注情况下准确率达94%(用药提取),停药识别达78.1%。
  • 开源模型表现接近闭源,适合大规模部署,少样本学习可进一步提升性能。

从电子健康记录(EHR)中识别药物停用对患者安全至关重要,但信息常隐藏于非结构化文本中。本研究评估了先进开源与专有大语言模型(LLMs)在从EHR笔记中提取药物及分类药物状态的能力,重点考察其在无需人工标注情况下的可扩展性。我们收集了来自不同来源的三个EHR数据集构建评估基准,测试了12个先进LLM并探索多种提示策略。系统比较了药物提取、药物状态分类及其联合任务的表现。结果显示,LLMs在药物提取与停药分类上表现良好。GPT-4o在零样本设置下各项任务平均F1得分最高——药物提取达94.0%,停药分类达78.1%,联合任务为72.7%。开源模型表现紧随其后,Llama-3.1-70B-Instruct在MIV-Med数据集上药物状态分类达68.7%,在Re-CASI和MIV-Med数据集上联合任务分别达76.2%和60.2%。医学专用模型表现低于通用领域先进模型。少样本学习普遍提升性能,而思维链(CoT)推理效果不一。结果表明,LLMs在无标注条件下具有强大的药物提取与停药识别潜力,开源模型可作为闭源系统的可扩展替代方案,少样本学习能进一步增强其能力。

原文摘要 · Abstract (English)

Identifying medication discontinuations in electronic health records (EHRs) is vital for patient safety but is often hindered by information being buried in unstructured notes. This study aims to evaluate the capabilities of advanced open-sourced and proprietary large language models (LLMs) in extracting medications and classifying their medication status from EHR notes, focusing on their scalability on medication information extraction without human annotation. We collected three EHR datasets from diverse sources to build the evaluation benchmark. We evaluated 12 advanced LLMs and explored multiple LLM prompting strategies. Performance on medication extraction, medication status classification, and their joint task (extraction then classification) was systematically compared across all experiments. We found that LLMs showed promising performance on the medication extraction and discontinuation classification from EHR notes. GPT-4o consistently achieved the highest average F1 scores in all tasks under zero-shot setting - 94.0% for medication extraction, 78.1% for discontinuation classification, and 72.7% for the joint task. Open-sourced models followed closely, Llama-3.1-70B-Instruct achieved the highest performance in medication status classification on the MIV-Med dataset (68.7%) and in the joint task on both the Re-CASI (76.2%) and MIV-Med (60.2%) datasets. Medical-specific LLMs demonstrated lower performance compared to advanced general-domain LLMs. Few-shot learning generally improved performance, while CoT reasoning showed inconsistent gains. LLMs demonstrate strong potential for medication extraction and discontinuation identification on EHR notes, with open-sourced models offering scalable alternatives to proprietary systems and few-shot can further improve LLMs' capability.

医疗AI大模型药物识别零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。