用轻量级LoRA微调胃肠道疾病识别模型，节省显存且效果更优
Parameter Efficient Fine-tuning for Domain-specific Gastrointestinal Disease Recognition
- 引入低秩适配模块（LoRA），仅训练少量参数来调整预训练模型
- 在胃肠道疾病识别任务中，精度优于全参数微调，参数量减少90%以上
- 适合资源有限但需跨源医疗图像分析的研究者使用
尽管基于预训练基础模型的医学图像分析取得进展，跨源图像分布偏移问题仍普遍存在。传统做法是对每个数据源单独训练模型，但全量微调大型预训练模型会带来高昂的存储成本。为此，本文提出采用低秩适配（LoRA）模块进行下游分类任务微调。LoRA通过学习轻量级的低秩矩阵，对预训练权重进行扰动以优化特定任务。在胃肠道疾病识别任务中，该方法显著优于端到端微调，在保持高精度的同时大幅提升了参数效率。代码已开源：github.com/sanjay931/peft-gi-recognition。
原文摘要 · Abstract (English)
Despite recent advancements in the field of medical image analysis with the use of pretrained foundation models, the issue of distribution shifts between cross-source images largely remains adamant. To circumvent that issue, investigators generally train a separate model for each source. However, this method becomes expensive when we fully fine-tune pretrained large models for a single dataset, as we must store multiple copies of those models. Thus, in this work, we propose using a low-rank adaptation (LoRA) module for fine-tuning downstream classification tasks. LoRAs learn lightweight task-specific low-rank matrices that perturb pretrained weights to optimize those downstream tasks. For gastrointestinal tract diseases, they exhibit significantly better results than end-to-end finetuning with improved parameter efficiency. Code is available at: github.com/sanjay931/peft-gi-recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。