分析CLIP模型在胸部X光分类中的公平性问题
Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification
- 用零样本和多种微调方法评估模型表现
- 微调提升准确率但公平性问题仍存
- 适合医疗AI伦理与公平性研究者参考
胸部影像对医学诊断至关重要，近年来基于图像-文本预训练的视觉语言模型（如CLIP）展现出提升诊断准确性的潜力。然而，这些模型最初并非为医学图像设计，尽管已有针对医学图像训练的CLIP类模型，其在不同人群和疾病类别上的公平性问题仍缺乏系统研究。本研究对应用于胸部X光分类的CLIP类模型进行全面公平性分析，采用零样本推理及线性探测、MLP、低秩适配（LoRA）和全量微调等多种技术，在不同患者群体和病种上评估性能与公平性。结果表明，虽微调可提升准确性，但公平性缺陷依然显著，凸显了在基础模型中引入公平性干预的必要性。
原文摘要 · Abstract (English)
X-ray imaging is pivotal in medical diagnostics, offering non-invasive insights into a range of health conditions. Recently, vision-language models, such as the Contrastive Language-Image Pretraining (CLIP) model, have demonstrated potential in improving diagnostic accuracy by leveraging large-scale image-text datasets. However, since CLIP was not initially designed for medical images, several CLIP-like models trained specifically on medical images have been developed. Despite their enhanced performance, issues of fairness - particularly regarding demographic attributes - remain largely unaddressed. In this study, we perform a comprehensive fairness analysis of CLIP-like models applied to X-ray image classification. We assess their performance and fairness across diverse patient demographics and disease categories using zero-shot inference and various fine-tuning techniques, including Linear Probing, Multilayer Perceptron (MLP), Low-Rank Adaptation (LoRA), and full fine-tuning. Our results indicate that while fine-tuning improves model accuracy, fairness concerns persist, highlighting the need for further fairness interventions in these foundational models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。