用ChatGPT自动解析医疗处方中的自由文本,提升可读性与准确性。
Zero- and Few-shot Named Entity Recognition and Text Expansion in Medication Prescriptions using ChatGPT
- 通过提示工程实现零样本和少样本命名实体识别与文本扩展。
- 命名实体识别F1达0.94,文本扩展F1达0.87,优于零样本基线。
- 少样本提示有效减少幻觉,适合医疗安全关键场景使用。
医疗处方常以自由文本形式存在,混合使用多种语言、本地品牌名及非标准化格式和缩写。大语言模型(LLMs)在响应提示生成文本方面表现优异。本研究利用ChatGPT 3.5自动结构化并扩展出院记录中的药物陈述,使结果对人和机器更易理解。采用零样本和少样本设置,结合不同提示策略进行命名实体识别(NER)与文本扩展(EX)任务。100条药物陈述经人工标注与整理。NER性能采用严格匹配和部分匹配评估;文本扩展由两名专家基于语义等价性判断,使用精确率、召回率与F1分数衡量。结果显示,最佳提示在测试集上达到平均F1 0.94;少样本提示在文本扩展任务中表现最优,平均F1为0.87。研究证明,利用ChatGPT处理自由文本药物陈述,在命名实体识别与文本扩展任务中均表现良好。相较于零样本基线,少样本方法能有效防止系统产生幻觉,这对处理高安全要求的药物数据至关重要。
原文摘要 · Abstract (English)
Introduction: Medication prescriptions are often in free text and include a mix of two languages, local brand names, and a wide range of idiosyncratic formats and abbreviations. Large language models (LLMs) have shown promising ability to generate text in response to input prompts. We use ChatGPT 3.5 to automatically structure and expand medication statements in discharge summaries and thus make them easier to interpret for people and machines. Methods: Named-entity Recognition (NER) and Text Expansion (EX) are used in a zero- and few-shot setting with different prompt strategies. 100 medication statements were manually annotated and curated. NER performance was measured by using strict and partial matching. For the task EX, two experts interpreted the results by assessing semantic equivalence between original and expanded statements. The model performance was measured by precision, recall, and F1 score. Results: For NER, the best-performing prompt reached an average F1 score of 0.94 in the test set. For EX, the few-shot prompt showed superior performance among other prompts, with an average F1 score of 0.87. Conclusion: Our study demonstrates good performance for NER and EX tasks in free-text medication statements using ChatGPT. Compared to a zero-shot baseline, a few-shot approach prevented the system from hallucinating, which would be unacceptable when processing safety-relevant medication data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。