用共享提示提升临床NLP迁移效率，参数少于0.05%仍更优。
A Parameter-Efficient Transfer Learning Approach through Multitask Prompt Distillation and Decomposition for Clinical NLP
- 从21个临床任务中提炼共享元提示，统一适配新任务。
- 仅需不足0.05%可训练参数，性能超LoRA 1.5~1.7%，超单任务提示6.1~6.6%。
- 适合资源受限场景下的多任务临床NLP部署，尤其擅长推理类任务。
现有基于提示的微调方法通常独立学习各任务提示，在部署多个临床自然语言处理系统时带来显著计算与存储开销。本文提出一种多任务提示蒸馏与分解框架，从21个多样化的临床源任务中学习单一共享元提示，并在未见目标任务上仅用少于0.05%的可训练参数进行适配。在五个临床NLP任务类型（命名实体识别、关系抽取、问答、自然语言推断和摘要）上，使用三个主干模型（LLaMA 3.1 8B、Meditron3 8B、gpt-oss 20B）对10个保留目标数据集进行评估，该框架在所有任务中均优于LoRA 1.5~1.7%，且显著超越单任务提示调优6.1~6.6%。gpt-oss 20B模型表现最佳，尤其在临床推理任务中优势明显。共享提示表示具有强零样本与少样本泛化能力，表明其优异的迁移性。
原文摘要 · Abstract (English)
Existing prompt-based fine-tuning methods typically learn task-specific prompts independently, imposing significant computing and storage overhead at scale when deploying multiple clinical natural language processing (NLP) systems. We present a multitask prompt distillation and decomposition framework that learns a single shared metaprompt from 21 diverse clinical source tasks and adapts it to unseen target tasks with fewer than 0.05% trainable parameters. Evaluated across five clinical NLP task types (named entity recognition, relation extraction, question answering, natural language inference, and summarization) on 10 held-out target datasets using three backbone models (LLaMA 3.1 8B, Meditron3 8B, gpt-oss 20B), our framework consistently outperforms LoRA by 1.5~1.7% despite using orders of magnitude fewer parameters, and exceeds single-task prompt tuning by 6.1~6.6%. The gpt-oss 20B model achieves the highest overall performance, particularly on clinical reasoning tasks. The strong zero- and few-shot performance demonstrates better transferability of the shared prompt representation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。