轻量化大模型在病理图像分析中的高效微调方法
EFCM: Efficient Fine-tuning on Compressed Models for deployment of large models in medical image analysis
- 分两阶段：无监督特征蒸馏+滑块级微调，解决高分辨率图像梯度回传难题
- 在TCGA-NSCLC和TCGA-BRCA上比原模型提升4.33%准确率、5.2%AUC
- 适合医疗影像部署场景，尤其适用于资源受限环境下的大模型应用
医学深度学习大模型在病理图像分析与诊断中表现优异，但参数量过大导致内存占用高、推理延迟严重。知识蒸馏可缓解此问题，但因病理图像分辨率高且标签为滑块级，难以实现学生模型的梯度反传。本文提出高效压缩模型微调框架EFCM，包含两个阶段：无监督特征蒸馏与滑块级微调。蒸馏阶段引入带有TransScan模块的特征投影蒸馏（FPD），自适应调整感受野以增强学生模型的知识吸收能力。微调阶段对比三种策略：Reuse CLAM、Retrain CLAM与端到端训练CLAM（ETC）。在11个下游数据集上评估三个大模型（RETFound用于视网膜、MRM用于胸部X光、BROW用于组织病理）的表现。实验表明，EFCM显著提升了处理滑块级病理图像任务的准确性与效率，有效解决了大模型部署难题。具体而言，在TCGA-NSCLC和TCGA-BRCA数据集上，相比原模型BROW，准确率提升4.33%，AUC提升5.2%。模型推理效率分析显示该方法具有较高运行效率。
原文摘要 · Abstract (English)
The recent development of deep learning large models in medicine shows remarkable performance in medical image analysis and diagnosis, but their large number of parameters causes memory and inference latency challenges. Knowledge distillation offers a solution, but the slide-level gradients cannot be backpropagated for student model updates due to high-resolution pathological images and slide-level labels. This study presents an Efficient Fine-tuning on Compressed Models (EFCM) framework with two stages: unsupervised feature distillation and fine-tuning. In the distillation stage, Feature Projection Distillation (FPD) is proposed with a TransScan module for adaptive receptive field adjustment to enhance the knowledge absorption capability of the student model. In the slide-level fine-tuning stage, three strategies (Reuse CLAM, Retrain CLAM, and End2end Train CLAM (ETC)) are compared. Experiments are conducted on 11 downstream datasets related to three large medical models: RETFound for retina, MRM for chest X-ray, and BROW for histopathology. The experimental results demonstrate that the EFCM framework significantly improves accuracy and efficiency in handling slide-level pathological image problems, effectively addressing the challenges of deploying large medical models. Specifically, it achieves a 4.33% increase in ACC and a 5.2% increase in AUC compared to the large model BROW on the TCGA-NSCLC and TCGA-BRCA datasets. The analysis of model inference efficiency highlights the high efficiency of the distillation fine-tuning method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。