用GPT-4标注数据训练模型,自动分类医疗消息减轻医生负担
OPTIC: Optimizing Patient-Provider Triaging & Improving Communications in Clinical Operations using GPT-4 Data Labeling and Model Distillation
- 用GPT-4生成高质量标签,训练BERT模型自动分类患者消息
- 模型准确率达88.85%,对81个主题识别超过80%准确率
- 已在真实医院系统部署,适合需要优化医患沟通的医疗机构
新冠疫情加速了远程医疗和电子病历门户中的患者消息(患者医疗咨询请求,PMARs)使用。尽管提升了患者就医便利性,但消息量激增也加重了医务人员负担。本研究开发了OPTIC系统,通过GPT-4进行数据标注,利用BERT进行模型压缩,实现消息自动分类。基于约翰霍普金斯医学中心2020年1月至6月的405,487条患者消息数据,采用GPT-4提示工程生成高质量标签,训练出可区分‘行政’与‘临床’类消息的BERT模型。测试集上模型准确率达88.85%,敏感度88.29%,特异度89.38%,F1值0.8842。BERTopic分析识别出81个独立主题,对其中58个主题分类准确率超80%。系统已成功集成至Epic Nebula云平台,在实际医疗环境中验证了其有效性。
原文摘要 · Abstract (English)
The COVID-19 pandemic has accelerated the adoption of telemedicine and patient messaging through electronic medical portals (patient medical advice requests, or PMARs). While these platforms enhance patient access to healthcare, they have also increased the burden on healthcare providers due to the surge in PMARs. This study seeks to develop an efficient tool for message triaging to reduce physician workload and improve patient-provider communication. We developed OPTIC (Optimizing Patient-Provider Triaging & Improving Communications in Clinical Operations), a powerful message triaging tool that utilizes GPT-4 for data labeling and BERT for model distillation. The study used a dataset of 405,487 patient messaging encounters from Johns Hopkins Medicine between January and June 2020. High-quality labeled data was generated through GPT-4-based prompt engineering, which was then used to train a BERT model to classify messages as "Admin" or "Clinical." The BERT model achieved 88.85% accuracy on the test set validated by GPT-4 labeling, with a sensitivity of 88.29%, specificity of 89.38%, and an F1 score of 0.8842. BERTopic analysis identified 81 distinct topics within the test data, with over 80% accuracy in classifying 58 topics. The system was successfully deployed through Epic's Nebula Cloud Platform, demonstrating its practical effectiveness in healthcare settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。