arXiv:2608.12086cs.CVcs.LG2026-08

发现医学图像模型依赖扫描仪噪声等捷径,影响诊断可靠性。

Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP

论文配图:Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP
图 1 · 摘自论文原文
  • 用线性探针分析模型各层特征,追踪捷径形成位置。
  • 最终分类器AUROC高但校准差,表明结果不可靠。
  • 适合关注医疗AI模型可信度的研究者和临床开发者。

基于对比语言-图像预训练(CLIP)的视觉语言模型在医疗人工智能中已达顶尖水平,但近期研究揭示其仍易受捷径影响。本文探究真实世界中捷径如何在医学CLIP模型MedCLIP及其冻结的ResNet-50视觉编码器中表现。我们在ResNet-50的中间层部署17个线性分类探针,分别在三个数据集配置与目标上训练:NIH-CXR14(气胸)、PadChest(心大和气胸)。通过子组校准与逐层置信度曲线分析模型行为。结果发现,最终探针虽达到高AUROC,但校准性能差。逐层分析显示,局部捷径(如引流管)出现在深层,而扩散型捷径(如特定扫描仪噪声)更早出现,与先前研究一致。手动图像分析揭示了NIH-CXR14和PadChest均存在数据质量问题。研究强调,即使顶尖模型仍受捷径影响,高质量、精细标注数据对得出可靠结论至关重要。代码已开源于GitHub:https://github.com/nikodice4/MedCLIP_shortcuts。

原文摘要 · Abstract (English)

Vision-language models, such as contrastive language-image pre-training (CLIP)-based approaches, have reached state-of-the-art (SOTA) results in medical artificial intelligence. However, recent work reveals that CLIP-based models remain vulnerable to shortcuts. We investigate how real-world shortcuts manifest across different layers of the medical CLIP-based model, MedCLIP, and its vision encoder, a frozen ResNet-50. We attach 17 linear classification probes to the intermediate layers of the ResNet-50 and train them on three different dataset configurations and targets: NIH-CXR14 (pneumothorax) and PadChest (cardiomegaly and pneumothorax). This setup allows us to observe model behaviour during evaluation using subgroup-based calibration and layer-wise confidence curves. We find that the final linear probes achieve a high AUROC but poor calibration in the models. The layer-wise confidence analyses suggest that shortcuts emerge at different depths. Patterns consistent with localised shortcuts, such as drains, appear at later layers, while patterns consistent with diffuse shortcuts, such as scanner-specific noise patterns, emerge earlier, aligning with previous work. Finally, we conduct a manual analysis of the images, which reveals data quality issues in both NIH-CXR14 and PadChest. Our findings underscore that even SOTA models remain vulnerable to shortcuts, and the need for high-quality and well-annotated datasets to draw solid conclusions. Code can be found on our GitHub: https://github.com/nikodice4/MedCLIP_shortcuts.

医学影像模型捷径数据质量可视化分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。