用轻量适配器+医学大模型嵌入,实现高效准确的放射影像分类。
From Embeddings to Accuracy: Comparing Foundation Models for Radiographic Classification
- 用7种大模型嵌入+简单适配器训练放射影像分类器
- MedImageInsight+SVM/MLP达93.1% mAUC,最优
- 模型轻量,训练分钟级,适合临床部署
基础模型为多种任务提供稳健的嵌入表示,包括医学影像分析。本文评估了七种通用及医学专用基础模型(如DenseNet121、BiomedCLIP、MedImageInsight、Rad-DINO、CXR-Foundation)在多类放射影像分类中生成的嵌入表示,并用于训练轻量级适配器。基于包含8,842张放射影像的七类数据集,采用K-近邻、逻辑回归、SVM、随机森林和MLP等算法训练适配器。结果显示,MedImageInsight嵌入配合SVM或MLP适配器取得最高均值AUC(mAUC)93.1%,显著优于其他模型,包括MedSigLIP+MLP(91.0%)、Rad-DINO+SVM(90.7%)和CXR-Foundation+逻辑回归(88.6%)。相比之下,BiomedCLIP(82.8%)和Med-Flamingo(78.5%)表现较差。关键的是,这些轻量适配器训练仅需数分钟,推理可在CPU上秒级完成,具备临床实用性。对表现最佳的MedImageInsight适配器进行公平性分析显示,性别间性能差异小于1.8%,年龄组标准差小于1.4%,无显著统计差异。结果表明,特定医学大模型嵌入结合轻量适配器,可构建准确、高效且公平的诊断工具。
原文摘要 · Abstract (English)
Foundation models provide robust embeddings for diverse tasks, including medical imaging. We evaluate embeddings from seven general and medical-specific foundation models (e.g., DenseNet121, BiomedCLIP, MedImageInsight, Rad-DINO, CXR-Foundation) for training lightweight adapters in multi-class radiography classification. Using a dataset of 8,842 radiographs across seven classes, we trained adapters with algorithms like K-Nearest Neighbors, logistic regression, SVM, random forest, and MLP. The combination of MedImageInsight embeddings with an SVM or MLP adapter achieved the highest mean area under the curve (mAUC) of 93.1%. This performance was statistically superior to other models, including MedSigLIP with an MLP (91.0%), Rad-DINO with an SVM (90.7%), and CXR-Foundation with logistic regression (88.6%). In contrast, models like BiomedCLIP (82.8%) and Med-Flamingo (78.5%) showed lower performance. Crucially, these lightweight adapters are computationally efficient, training in minutes and performing inference in seconds on a CPU, making them practical for clinical use. A fairness analysis of the top-performing MedImageInsight adapter revealed minimal performance disparities across patient gender (within 1.8%) and age groups (std. dev < 1.4%), with no significant statistical differences. These findings confirm that embeddings from specialized foundation models, particularly MedImageInsight, can power accurate, efficient, and equitable diagnostic tools using simple, lightweight adapters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。