LoRA在医学摘要任务中以极少量参数超越全量微调，节省资源且效果更好。
Parameter-Efficient Fine-Tuning for Medical Text Summarization: A Comparative Study of Lora, Prompt Tuning, and Full Fine-Tuning
- 用低秩矩阵替换部分权重，仅更新0.6%参数实现高效微调
- LoRA在Flan-T5-Large上达43.52的ROUGE-1，优于全量微调的40.67
- 适合医疗、资源受限场景下快速部署大模型摘要系统
针对医学文本摘要等特定领域任务，微调大语言模型需大量计算资源。参数高效微调（PEFT）方法通过仅更新少量参数提供可行替代方案。本文在PubMed医学摘要数据集上，对比了低秩适配（LoRA）、提示微调（Prompt Tuning）与全量微调在Flan-T5模型族上的表现。通过多随机种子实验，发现LoRA始终优于全量微调：在Flan-T5-Large上，仅用0.6%可训练参数即达43.52±0.18的ROUGE-1，而全量微调为40.67±0.21。敏感性分析考察了LoRA秩和提示词数量的影响。结果表明低秩约束具有有益正则化作用，挑战了必须全量更新参数的假设。代码已开源。
原文摘要 · Abstract (English)
Fine-tuning large language models for domain-specific tasks such as medical text summarization demands substantial computational resources. Parameter-efficient fine-tuning (PEFT) methods offer promising alternatives by updating only a small fraction of parameters. This paper compares three adaptation approaches-Low-Rank Adaptation (LoRA), Prompt Tuning, and Full Fine-Tuning-across the Flan-T5 model family on the PubMed medical summarization dataset. Through experiments with multiple random seeds, we demonstrate that LoRA consistently outperforms full fine-tuning, achieving 43.52 +/- 0.18 ROUGE-1 on Flan-T5-Large with only 0.6% trainable parameters compared to 40.67 +/- 0.21 for full fine-tuning. Sensitivity analyses examine the impact of LoRA rank and prompt token count. Our findings suggest the low-rank constraint provides beneficial regularization, challenging assumptions about the necessity of full parameter updates. Code is available at https://github.com/eracoding/llm-medical-summarization
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。