用相似度证据校准视觉语言模型,提升医学主动学习的准确性和可解释性。
Similarity-as-Evidence: Calibrating Overconfident VLMs for Interpretable and Label-Efficient Medical Active Learning
- 将图文相似度重构成证据,用狄利克雷分布量化不确定性
- 在20%标注预算下实现82.57%的宏观准确率,领先现有方法
- 区分罕见病和模糊诊断,选择理由可临床解释,适合医疗场景
主动学习通过筛选最有信息量的样本降低医学影像标注成本,但在标签数据稀缺时面临冷启动问题。视觉语言模型虽可通过零样本预测缓解此问题,但其温度缩放的softmax输出将图文相似度视为确定性得分,忽略内在不确定性,导致过度自信。这种过度自信会误导样本选择,浪费标注资源于低效案例。为此,本文提出相似度作为证据(SaE)框架,引入相似度证据头(SEH),将相似度向量重新解释为证据,并对标签参数化狄利克雷分布。相比强制高置信度的softmax,狄利克雷形式显式量化证据缺失(空虚)与证据冲突(不一致),有效缓解由刚性归一化引发的过度自信。基于此,SaE采用双因子选择策略:早期优先选择高空虚样本(如罕见病)以确保覆盖,后期优先选择高不一致样本(如模糊诊断)以细化边界,提供可临床解释的选择依据。在十个公开医学影像数据集上,使用20%标注预算,SaE达到82.57%的最新宏观平均准确率;在代表性BTMRI数据集上,其校准性能更优,负对数似然(NLL)为0.425。
原文摘要 · Abstract (English)
Active Learning (AL) reduces annotation costs in medical imaging by selecting only the most informative samples for labeling, but suffers from cold-start when labeled data are scarce. Vision-Language Models (VLMs) address the cold-start problem via zero-shot predictions, yet their temperature-scaled softmax outputs treat text-image similarities as deterministic scores while ignoring inherent uncertainty, leading to overconfidence. This overconfidence misleads sample selection, wasting annotation budgets on uninformative cases. To overcome these limitations, the Similarity-as-Evidence (SaE) framework calibrates text-image similarities by introducing a Similarity Evidence Head (SEH), which reinterprets the similarity vector as evidence and parameterizes a Dirichlet distribution over labels. In contrast to a standard softmax that enforces confident predictions even under weak signals, the Dirichlet formulation explicitly quantifies lack of evidence (vacuity) and conflicting evidence (dissonance), thereby mitigating overconfidence caused by rigid softmax normalization. Building on this, SaE employs a dual-factor acquisition strategy: high-vacuity samples (e.g., rare diseases) are prioritized in early rounds to ensure coverage, while high-dissonance samples (e.g., ambiguous diagnoses) are prioritized later to refine boundaries, providing clinically interpretable selection rationales. Experiments on ten public medical imaging datasets with a 20% label budget show that SaE attains state-of-the-art macro-averaged accuracy of 82.57%. On the representative BTMRI dataset, SaE also achieves superior calibration, with a negative log-likelihood (NLL) of 0.425.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。