小模型也能搞定医疗NLP,效果甚至超过大模型。
Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian
- 用百亿参数小模型配合微调,性能优于大模型。
- 最佳配置比320亿参数大模型平均高9.2分。
- 适合资源有限的医院或研究者快速部署。
大型语言模型在医疗自然语言处理任务中表现优异,但计算成本高,限制了实际应用。本文研究约十亿参数的小型模型是否能在医疗任务中保持竞争力。我们评估了Llama-3、Gemma-3和Qwen3三个系列模型在20个临床NLP任务中的表现,涵盖命名实体识别、关系抽取、病例报告填写、问答和论点挖掘。系统比较了推理阶段(少样本提示、约束解码)和训练阶段(监督微调、持续预训练)的多种适配策略。结果显示,微调是最有效的方案,而少样本提示与约束解码的组合则为低资源场景提供有力替代。最佳配置基于Qwen3-1.7B,在平均得分上比Qwen3-32B高出9.2分。我们公开了所有可用的意大利语医疗NLP数据集,以及表现最优的模型。此外,还发布了来自意大利某急诊科的1.26亿词医疗文本数据,以及1.75亿词用于持续预训练的多源数据。
原文摘要 · Abstract (English)
Large Language Models (LLMs) consistently excel in diverse medical Natural Language Processing (NLP) tasks, yet their substantial computational requirements often limit deployment in real-world healthcare settings. In this work, we investigate whether "small" LLMs (around one billion parameters) can effectively perform medical tasks while maintaining competitive accuracy. We evaluate models from three major families-Llama-3, Gemma-3, and Qwen3-across 20 clinical NLP tasks among Named Entity Recognition, Relation Extraction, Case Report Form Filling, Question Answering, and Argument Mining. We systematically compare a range of adaptation strategies, both at inference time (few-shot prompting, constraint decoding) and at training time (supervised fine-tuning, continual pretraining). Fine-tuning emerges as the most effective approach, while the combination of few-shot prompting and constraint decoding offers strong lower-resource alternatives. Our results show that small LLMs can match or even surpass larger baselines, with our best configuration based on Qwen3-1.7B achieving an average score +9.2 points higher than Qwen3-32B. We release a comprehensive collection of all the publicly available Italian medical datasets for NLP tasks, together with our top-performing models. Furthermore, we release an Italian dataset of 126M words from the Emergency Department of an Italian Hospital, and 175M words from various sources that we used for continual pre-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。