对比多种大模型在病患信息抽取中的表现,找最优架构与微调策略。
A Study of Large Language Models for Patient Information Extraction: Model Architecture, Fine-Tuning Strategy, and Multi-task Instruction Tuning
- 测试编码器与解码器类大模型在临床文本中的表现差异
- 提示微调比全量微调更高效,且在少样本下仍有效
- 多任务指令微调提升零样本与少样本学习能力,适合医疗场景
自然语言处理是提取临床叙述中关键患者信息以支持医疗应用的关键技术。大语言模型的快速发展已革新了多个临床领域的NLP任务,但其在患者信息抽取任务中的最佳应用仍有待探索。本研究评估大语言模型在患者信息抽取中的有效性,重点关注模型架构、微调策略及多任务指令微调技术,旨在构建稳健且可泛化的患者信息抽取系统。研究涵盖编码器型(BERT、GatorTron)与解码器型(GatorTronGPT、Llama 3.1、GatorTronLlama)大模型,在五个数据集上进行基准测试。比较了传统全量微调与基于提示的参数高效微调(PEFT)。通过多任务指令微调框架,在四个数据集上结合两类任务,采用留一数据集外策略评估零样本与少样本学习性能。
原文摘要 · Abstract (English)
Natural language processing (NLP) is a key technology to extract important patient information from clinical narratives to support healthcare applications. The rapid development of large language models (LLMs) has revolutionized many NLP tasks in the clinical domain, yet their optimal use in patient information extraction tasks requires further exploration. This study examines LLMs' effectiveness in patient information extraction, focusing on LLM architectures, fine-tuning strategies, and multi-task instruction tuning techniques for developing robust and generalizable patient information extraction systems. This study aims to explore key concepts of using LLMs for clinical concept and relation extraction tasks, including: (1) encoder-only or decoder-only LLMs, (2) prompt-based parameter-efficient fine-tuning (PEFT) algorithms, and (3) multi-task instruction tuning on few-shot learning performance. We benchmarked a suite of LLMs, including encoder-based LLMs (BERT, GatorTron) and decoder-based LLMs (GatorTronGPT, Llama 3.1, GatorTronLlama), across five datasets. We compared traditional full-size fine-tuning and prompt-based PEFT. We explored a multi-task instruction tuning framework that combines both tasks across four datasets to evaluate the zero-shot and few-shot learning performance using the leave-one-dataset-out strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。