用大模型从尸检报告中自动识别药物,提升毒品过量死亡监测的准确性和速度。
Improving Drug Identification in Overdose Death Surveillance using Large Language Models
- 用微调的BioClinicalBERT模型从自由文本中提取药物信息
- 内部测试F1超0.998,外部验证仍达0.966,远超传统方法
- 适合公共卫生部门做实时毒情监测,也适用于临床数据挖掘
美国药物相关死亡率持续上升,主要由芬太尼驱动,亟需及时准确的死亡监测。但关键数据常隐藏在自由文本的法医报告中,人工转录为ICD-10编码存在延迟和信息丢失。本文基于2020年来自多个美国司法管辖区的35,433条死亡记录训练模型,并使用2023–2024年新采集的3,335条独立数据集进行外部验证。评估了多种NLP方法对未结构化死亡证明文本中的具体药物参与情况进行分类的能力,包括传统单/多标签分类器,以及微调的编码器模型(如BERT、BioClinicalBERT)和当代解码器模型(如Qwen 3、Llama 3)。采用宏平均F1分数评估性能,并计算95%置信区间以量化不确定性。微调后的BioClinicalBERT在内部测试集上表现接近完美(宏F1≥0.998),外部验证显示其稳健性(宏F1=0.966),显著优于传统机器学习、通用领域BERT及各类解码器模型。结果表明,尤其是微调的临床类语言模型,可为从自由文本报告中进行过量死亡分类提供高精度、可扩展的解决方案,大幅加速监测流程,克服人工编码局限,支持近实时发现新兴物质滥用趋势。
原文摘要 · Abstract (English)
The rising rate of drug-related deaths in the United States, largely driven by fentanyl, requires timely and accurate surveillance. However, critical overdose data are often buried in free-text coroner reports, leading to delays and information loss when coded into ICD (International Classification of Disease)-10 classifications. Natural language processing (NLP) models may automate and enhance overdose surveillance, but prior applications have been limited. A dataset of 35,433 death records from multiple U.S. jurisdictions in 2020 was used for model training and internal testing. External validation was conducted using a novel separate dataset of 3,335 records from 2023-2024. Multiple NLP approaches were evaluated for classifying specific drug involvement from unstructured death certificate text. These included traditional single- and multi-label classifiers, as well as fine-tuned encoder-only language models such as Bidirectional Encoder Representations from Transformers (BERT) and BioClinicalBERT, and contemporary decoder-only large language models such as Qwen 3 and Llama 3. Model performance was assessed using macro-averaged F1 scores, and 95% confidence intervals were calculated to quantify uncertainty. Fine-tuned BioClinicalBERT models achieved near-perfect performance, with macro F1 scores >=0.998 on the internal test set. External validation confirmed robustness (macro F1=0.966), outperforming conventional machine learning, general-domain BERT models, and various decoder-only large language models. NLP models, particularly fine-tuned clinical variants like BioClinicalBERT, offer a highly accurate and scalable solution for overdose death classification from free-text reports. These methods can significantly accelerate surveillance workflows, overcoming the limitations of manual ICD-10 coding and supporting near real-time detection of emerging substance use trends.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。