用大模型自动给处方打药学编码,保护隐私还省人力
Zero-Shot ATC Coding with Large Language Models for Clinical Assessments
- 把药物编码任务拆解成逐层推理的结构化提取,引导大模型理解分类体系
- 在真实医疗数据上达到60%准确率(小模型)和78%(GPT-4o),效果接近人工
- 证明小模型经微调后可替代大模型,适合医院本地部署
在安大略健康与InterRAI加拿大机构中,手动为处方记录分配解剖治疗化学(ATC)编码是医疗研究与运营中的重大瓶颈,需大量专家投入。为实现自动化并保障数据隐私,我们提出一种基于本地部署大语言模型(LLM)的实用方法。受国际疾病分类(ICD)编码自动化的启发,本方法将ATC编码视为分层信息抽取任务,引导模型逐级解析ATC分类体系。我们以GPT-4o为准确率基准,重点开发适用于隐私敏感环境的开源Llama模型。在加拿大卫生部药品数据、RABBITS基准及安大略健康真实临床笔记上测试,该方法在使用GPT-4o时达到78%精确匹配准确率,在Llama 3.1 70B上达60%。通过引入药物定义进行知识增强,仅带来小幅准确率提升。进一步发现,微调后的Llama 3.1 8B可达到与零样本Llama 3.1 70B相当的性能,表明小模型也可实现高效编码。结果证明,在注重隐私的医疗环境中实现自动ATC编码是可行的,为未来部署奠定基础。
原文摘要 · Abstract (English)
Manual assignment of Anatomical Therapeutic Chemical (ATC) codes to prescription records is a significant bottleneck in healthcare research and operations at Ontario Health and InterRAI Canada, requiring extensive expert time and effort. To automate this process while maintaining data privacy, we develop a practical approach using locally deployable large language models (LLMs). Inspired by recent advances in automatic International Classification of Diseases (ICD) coding, our method frames ATC coding as a hierarchical information extraction task, guiding LLMs through the ATC ontology level by level. We evaluate our approach using GPT-4o as an accuracy ceiling and focus development on open-source Llama models suitable for privacy-sensitive deployment. Testing across Health Canada drug product data, the RABBITS benchmark, and real clinical notes from Ontario Health, our method achieves 78% exact match accuracy with GPT-4o and 60% with Llama 3.1 70B. We investigate knowledge grounding through drug definitions, finding modest improvements in accuracy. Further, we show that fine-tuned Llama 3.1 8B matches zero-shot Llama 3.1 70B accuracy, suggesting that effective ATC coding is feasible with smaller models. Our results demonstrate the feasibility of automatic ATC coding in privacy-sensitive healthcare environments, providing a foundation for future deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。