用低秩适配让大模型高效落地医疗影像，零样本分类性能显著提升
Scaling Down to Scale Up: Towards Operationally-Efficient and Deployable Clinical Models via Cross-Modal Low-Rank Adaptation for Medical Vision-Language Models
- 在视觉与文本编码器中插入低秩矩阵，仅训练167万参数（0.38%）实现高效微调
- 零样本分类下平均AUROC提升至68.9%，准确率和宏F1分别达73.6%和36.9%
- 适合资源受限场景下的临床部署，尤其适用于标注数据稀缺的医学任务
通过视觉-语言预训练构建的基础模型在多个图像领域展现出强大的零样本能力，但在体积化医学影像中的应用仍受限。本文提出MedCT-VLM：一种面向胸部CT的视觉-语言模型，基于参数高效的低秩适配（LoRA）方法，将大规模CT基础模型CT-CLIP用于下游临床任务。该模型在25,692例胸部CT数据上训练的对比学习模型基础上，通过在视觉和文本编码器的注意力层嵌入低秩分解矩阵，仅训练167万参数（占总参数4400万的0.38%），而非直接微调全部参数。在18种胸腔病理的零样本分类任务中，模型需在推理时将CT特征与未见文本提示对齐，无需任务特定训练。经过LoRA微调后，平均AUROC从61.3%提升至68.9%（+7.6个百分点），准确率由67.2%升至73.6%（+6.4个百分点），宏F1从32.1%增至36.9%（+4.8个百分点）。结果表明，参数高效方法能有效将大规模预训练知识迁移至下游医学影像任务，尤其在标签数据稀少的零样本场景中表现优异。
原文摘要 · Abstract (English)
Foundation models trained via vision-language pretraining have demonstrated strong zero-shot capabilities across diverse image domains, yet their application to volumetric medical imaging remains limited. We introduce MedCT-VLM: Medical CT Vision-Language Model, a parameter-efficient vision-language framework designed to adapt large-scale CT foundation models for downstream clinical tasks. MedCT-VLM uses a parameter-efficient approach to adapt CT-CLIP, a contrastive vision-language model trained on 25,692 chest CT volumes, for multi-label pathology classification using Low-Rank Adaptation (LoRA). Rather than fine-tuning the model's 440 M parameters directly, we insert low-rank decomposition matrices into attention layers of both vision and text encoders, training only 1.67M parameters (0.38\% of total). We evaluate on zero-shot classification across 18 thoracic pathologies, where the model must align CT embeddings with unseen text prompts at inference without task-specific training. LoRA fine-tuning improves mean AUROC from 61.3\% to 68.9\% (+7.6 pp), accuracy from 67.2\% to 73.6\% (+6.4 pp), and macro-F1 from 32.1\% to 36.9\% (+4.8 pp). These results demonstrate that parameter-efficient methods can effectively transfer large-scale pretraining to downstream medical imaging tasks, particularly for zero-shot scenarios where labeled data is scarce.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。