用轻量适配器实现医学影像多标签诊断的持续学习
RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification
- 冻结主干网络，仅训练简单适配器，提升效率
- 在三种增量场景下均避免灾难性遗忘，保持高精度
- 适合临床部署，兼顾隐私与计算效率
本文提出RE-tune，一种面向多标签胸部疾病诊断的医学视觉-语言模型增量微调方法。该方法冻结模型主干，仅在图像和文本编码器上训练轻量适配器，并通过设计正负文本提示引导大语言模型优化训练路径。我们在类增量、标签增量和数据增量三种真实场景下评估，结果表明生物医学视觉-语言模型具备天然的持续学习能力，能有效防止灾难性遗忘。RE-tune不仅实现精准多标签分类，还显著降低计算开销，保护患者隐私，适用于真实医疗环境中的广泛部署。
原文摘要 · Abstract (English)
In this paper we introduce RE-tune, a novel approach for fine-tuning pre-trained Multimodal Biomedical Vision-Language models (VLMs) in Incremental Learning scenarios for multi-label chest disease diagnosis. RE-tune freezes the backbones and only trains simple adaptors on top of the Image and Text encoders of the VLM. By engineering positive and negative text prompts for diseases, we leverage the ability of Large Language Models to steer the training trajectory. We evaluate RE-tune in three realistic incremental learning scenarios: class-incremental, label-incremental, and data-incremental. Our results demonstrate that Biomedical VLMs are natural continual learners and prevent catastrophic forgetting. RE-tune not only achieves accurate multi-label classification results, but also prioritizes patient privacy and it distinguishes itself through exceptional computational efficiency, rendering it highly suitable for broad adoption in real-world healthcare settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。