让病理图像模型一次预测所有区域，提升准确率。
Boosting Vision-Language Models for Histopathology Classification: Predict all at once
- 利用图文嵌入和区域关联性，实现整体推理而非单块独立判断。
- 处理十万级图像块仅需数秒，零样本分类准确率显著提升。
- 无需额外标注，在黑箱环境下高效适用，适合临床辅助诊断。
病理学视觉语言模型（VLM）展现出令人鼓舞的新应用和零样本性能。然而，现有方法将大病理切片分解为小块，仅进行归纳式分类——即对每个图像块的预测独立于其他块。本文提出一种新型归纳式方法，通过文本预测与块间亲和关系，利用这些大型VLM的强大零样本能力，无需任何额外标签即可增强模型表现。实验涵盖四个病理数据集和五种不同VLM。整个过程仅在嵌入空间中操作（即黑箱设置），效率极高，可在数秒内处理10⁵个图像块，并显著优于传统的归纳式零样本分类。代码已公开于https://github.com/FereshteShakeri/Histo-TransCLIP。
原文摘要 · Abstract (English)
The development of vision-language models (VLMs) for histo-pathology has shown promising new usages and zero-shot performances. However, current approaches, which decompose large slides into smaller patches, focus solely on inductive classification, i.e., prediction for each patch is made independently of the other patches in the target test data. We extend the capability of these large models by introducing a transductive approach. By using text-based predictions and affinity relationships among patches, our approach leverages the strong zero-shot capabilities of these new VLMs without any additional labels. Our experiments cover four histopathology datasets and five different VLMs. Operating solely in the embedding space (i.e., in a black-box setting), our approach is highly efficient, processing $10^5$ patches in just a few seconds, and shows significant accuracy improvements over inductive zero-shot classification. Code available at https://github.com/FereshteShakeri/Histo-TransCLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。