提出新方法提升3D CT影像零样本多异常检测的适应能力。
When Can Test-Time Adaptation Help Zero-Shot CT Vision-Language Models?

- 设计针对3D CT的无监督多标签测试时自适应方法,保留异常共现关系。
- 在多个任务上实现更稳定提升,尤其当基础模型已能区分异常存在与否时。
- 适合医疗影像领域需快速部署且标注稀缺的场景,如跨机构临床应用。
3D CT视觉语言模型(VLM)可在零样本条件下根据文本提示分类异常,适用于标签稀缺、临床任务频繁变化的跨机构部署场景。然而真实CT扫描常含多个共现异常,零样本多标签预测在分布偏移下的可靠性尚不明确。测试时自适应(TTA)可在无源数据或目标标注情况下更新模型,但现有方法主要面向自然图像多分类或2D医学分割,尚未解决零样本3D CT VLM的无监督多标签适配问题。本研究揭示TTA有效性的条件:体积输入必须保持编码器深度结构,基础表征需可迁移至目标队列;仅深度缩减即导致内部AUROC下降超过0.12。当基础模型已能区分异常存在与否时,我们提出首个针对性方法CARVE(Cardinality-Aware Retained-View Entropy),通过估计样本级正类数量\hat{k},优化前\hat{k}个输出以保留共现异常,并采用无需梯度的弱3D视图评分实现高效多视角自适应。在对比式CT-CLIP与解剖感知fVLM上,CARVE在多标签、三分类及二分类任务中均提供最一致的性能提升。
原文摘要 · Abstract (English)
3D CT vision-language models (VLMs) classify abnormalities from text prompts in a zero-shot manner, enabling cross-institution deployment where labels are scarce and clinical tasks shift faster than supervised models can be retrained. A real CT scan, however, typically contains several co-occurring abnormalities, and the reliability of zero-shot multi-label prediction under distribution shift remains poorly understood. Test-time adaptation (TTA) updates a model on unlabeled target scans without source data or target annotations, yet existing TTA methods target multi-class softmax prediction on natural images or 2D medical segmentation, and none addresses unsupervised multi-label adaptation for zero-shot 3D CT VLMs. We study when TTA helps zero-shot 3D CT VLMs. A controlled diagnostic analysis shows that TTA is conditional: the volumetric input must preserve the encoder's depth structure, and the base representation must transfer to the target cohort, with depth reduction alone lowering internal AUROC by more than 0.12. We then focus on the regime where the base model already separates present from absent abnormalities. We introduce CARVE (Cardinality-Aware Retained-View Entropy), the first TTA method for this setting. CARVE estimates a sample-specific positive-label cardinality $\hat{k}$, optimizes a top-$\hat{k}$ objective to preserve co-occurring abnormalities, and performs memory-efficient multi-view adaptation by scoring weak 3D views without gradients before updating on a retained subset. Across contrastive CT-CLIP and anatomy-aware fVLM, CARVE provides the most consistent improvements across multi-label, three-class, and binary CT tasks when the base model is already discriminative. These results establish multi-label TTA for zero-shot 3D CT VLMs as a distinct problem and CARVE as a cardinality-aware solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。