用多阶段大模型分析医院客服消息,自动提炼可操作的医疗洞察。
From Staff Messages to Actionable Insights: A Multi-Stage LLM Classification Framework for Healthcare Analytics
- 分阶段使用不同大模型识别消息主题与原因,提升分类效率。
- 最佳模型o3达78.4%加权F1,接近gpt-5的75.3%表现。
- 符合HIPAA要求,输出可转为可视化工具供医护决策使用。
医院呼叫中心是患者联系医疗机构的主要入口,同时产生大量工作人员消息,记录导航员处理请求及与院内部门沟通的过程。这些持续积累的文本数据蕴含潜在洞察,但传统监督学习需大量标注数据和调参。本文提出一种基于多阶段大语言模型(LLM)的分类框架,对工作人员消息进行主题识别与多类原因分类。评估了推理型、通用型及轻量级等多种模型,最优模型o3实现78.4%加权F1-score与79.2%准确率,紧随其后的是gpt-5(75.3%加权F1-score,76.2%准确率)。方法集成数据安全与HIPAA合规措施,处理后的输出被导入可视化决策支持工具,将原始消息转化为医护人员可直接使用的行动建议。该方案提升消息数据利用率,助力导航员培训优化,并推动患者体验与医疗质量改善。
原文摘要 · Abstract (English)
Hospital call centers serve as the primary contact point for patients within a hospital system. They also generate substantial volumes of staff messages as navigators process patient requests and communicate with the hospital offices following the established protocol restrictions and guidelines. This continuously accumulated large amount of text data can be mined and processed to retrieve insights; however, traditional supervised learning approaches require annotated data, extensive training, and model tuning. Large Language Models (LLMs) offer a paradigm shift toward more computationally efficient methodologies for healthcare analytics. This paper presents a multi-stage LLM-based framework that identifies staff message topics and classifies messages by their reasons in a multi-class fashion. In the process, multiple LLM types, including reasoning, general-purpose, and lightweight models, were evaluated. The best-performing model was o3, achieving 78.4% weighted F1-score and 79.2% accuracy, followed closely by gpt-5 (75.3% Weighted F1-score and 76.2% accuracy). The proposed methodology incorporates data security measures and HIPAA compliance requirements essential for healthcare environments. The processed LLM outputs are integrated into a visualization decision support tool that transforms the staff messages into actionable insights accessible to healthcare professionals. This approach enables more efficient utilization of the collected staff messaging data, identifies navigator training opportunities, and supports improved patient experience and care quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。