让医疗视觉语言模型更靠谱：通过提示学习提升置信度校准
Calibration-Aware Prompt Learning for Medical Vision-Language Models
- 设计可学习提示，加入校准目标，在少量标注数据下优化模型置信度
- 在5个医学影像数据集上，校准效果显著提升，准确率几乎不变
- 适合需要高可信预测的临床场景，如辅助诊断与决策支持
医疗视觉语言模型（Med-VLMs）通过大规模图文预训练在多种医学影像任务中表现出色。然而，其置信度校准问题尚未被充分探索，导致预测可能过度自信，影响临床信任与决策可靠性。为此，我们提出首个在提示调优过程中进行校准的框架CalibPrompt。该方法在标签稀缺条件下，优化一组可学习提示，引入精心设计的校准目标：首先，使用正则项使平滑准确率与模型预测置信度对齐；其次，引入角度分离损失，增强文本特征的相似性，以提升多模态模型置信度估计的可靠性。在四个公开可用的Med-VLM和五个多样化的医学影像数据集上的大量实验表明，CalibPrompt在不显著影响原始准确率的前提下，持续改善模型校准性能。代码已开源：https://github.com/iabh1shekbasu/CalibPrompt。
原文摘要 · Abstract (English)
Medical Vision-Language Models (Med-VLMs) have demonstrated remarkable performance across diverse medical imaging tasks by leveraging large-scale image-text pretraining. However, their confidence calibration is largely unexplored, and so remains a significant challenge. As such, miscalibrated predictions can lead to overconfident errors, undermining clinical trust and decision-making reliability. To address this, we introduce CalibPrompt, the first framework to calibrate Med-VLMs during prompt tuning. CalibPrompt optimizes a small set of learnable prompts with carefully designed calibration objectives under scarce labeled data regime. First, we study a regularizer that attempts to align the smoothed accuracy with the predicted model confidences. Second, we introduce an angular separation loss to maximize textual feature proximity toward improving the reliability in confidence estimates of multimodal Med-VLMs. Extensive experiments on four publicly available Med-VLMs and five diverse medical imaging datasets reveal that CalibPrompt consistently improves calibration without drastically affecting clean accuracy. Our code is available at https://github.com/iabh1shekbasu/CalibPrompt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。