arXiv:2605.15680cs.CLcs.LG2026-05被引 1

用少样本提示让大模型帮病人咨询自动分类,效果超传统方法。

Few-Shot Large Language Models for Actionable Triage Categorization of Online Patient Inquiries

论文配图:Few-Shot Large Language Models for Actionable Triage Categorization of Online Patient Inquiries
图 1 · 摘自论文原文
  • 用提示工程+少量标注样本训练大模型,实现医疗咨询分类。
  • 12样本提示下最强模型宏F1达0.475,优于最佳基线0.378。
  • 适合医疗系统做智能分流,但不能完全替代人工判断。

在线患者咨询通常非正式、不完整且在专业评估前提交,但仍需被正确引导至相应临床随访级别。本文将其建模为四类可操作分诊任务:自我护理、预约就诊、紧急医生审核或急诊转介。基于公开的HealthCareMagic-100K语料库,构建了包含300个经人工校准的黄金标准测试集、700个自动标注的银色训练集及40个少样本池。对比了TF-IDF与BioBERT等基线模型在银标数据上的表现,与六种提示式大语言模型在0、4、12样本条件下的性能。评估指标包括宏F1及安全相关指标(如急诊召回率、漏诊率、严重漏诊率)。最强模型Claude Haiku 4.5(12-shot)达到宏F1 0.475,高于最佳监督基线BioBERT(0.378),置信区间重叠。少样本提示与双模型一致性在不同类别中作用不同:自我护理判断较可靠,紧急医生审核则不可靠。结论是大模型可辅助分诊优先级判定与选择性人工审查,但不宜自主部署。

原文摘要 · Abstract (English)

Online patient inquiries are often informal, incomplete, and written before professional assessment, yet they must still be routed to an appropriate level of clinical follow-up. We study this as a four-class actionable triage task -- self-care, schedule-visit, urgent-clinician-review, or emergency-referral, and ask whether prompted large language models (LLMs) can support such routing under low-resource labeling conditions. Using the public HealthCareMagic-100K corpus, we construct a 300-example human calibrated gold evaluation set, a 700-example auto-labeled silver training set, and a 40-example few-shot pool. We compare Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT) baselines train on silver labels against six prompted LLMs under 0-shot, 4-shot, and 12-shot conditions respectively. Accordingly, we evaluate with macro-$F_1$ alongside safety-aware metrics, including emergency-recall, under-triage rate, and severe under-triage rate. The strongest LLM (Claude Haiku 4.5, 12-shot) reaches macro-$F_1$ 0.475, exceeding the best supervised baseline (BioBERT, 0.378) on point estimate, with overlapping confidence intervals. Few-shot prompting and two-model agreement help in label-dependent ways: self-care agreement is reliable, urgent-clinician-review is not. We conclude that LLMs can support triage prioritization and selective human review, but not autonomous deployment.

医疗AI少样本学习大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。