用视觉语言模型和提示学习，仅靠少量标注切片实现病理图像精准分类。
Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology
- 通过病理先验知识筛选关键组织区域，构建滑动窗级提示学习框架。
- 在少样本条件下，对多个真实病理数据集的分类准确率显著优于现有方法。
- 适合需要低标注成本的病理图像智能分析场景，尤其适用于小样本学习。
本文针对全切片图像（WSI）的少样本分类挑战，提出基于基础视觉语言模型（VLM）与滑动窗级提示学习的方法。由于WSI具有千兆像素级规模，传统多实例学习（MIL）依赖聚合函数从切片块表示中生成整体预测，需大量袋级标签进行训练。相比之下，基于VLM的方法虽能对齐切片块视觉嵌入与候选类别文本提示，但缺乏必要的病理先验知识。本方法通过语言模型引入病理先验，识别关键局部组织类型（切片块），并将其整合进VLM-based MIL框架。该方法有效对齐切片图像与组织类型，并仅使用每类少量标注的全切片图像进行提示学习微调。在真实病理WSI数据集上的实验及消融研究验证了其在少样本分类任务中的优越性能。代码已公开于https://github.com/LTS5/SLIP。
原文摘要 · Abstract (English)
In this paper, we address the challenge of few-shot classification in histopathology whole slide images (WSIs) by utilizing foundational vision-language models (VLMs) and slide-level prompt learning. Given the gigapixel scale of WSIs, conventional multiple instance learning (MIL) methods rely on aggregation functions to derive slide-level (bag-level) predictions from patch representations, which require extensive bag-level labels for training. In contrast, VLM-based approaches excel at aligning visual embeddings of patches with candidate class text prompts but lack essential pathological prior knowledge. Our method distinguishes itself by utilizing pathological prior knowledge from language models to identify crucial local tissue types (patches) for WSI classification, integrating this within a VLM-based MIL framework. Our approach effectively aligns patch images with tissue types, and we fine-tune our model via prompt learning using only a few labeled WSIs per category. Experimentation on real-world pathological WSI datasets and ablation studies highlight our method's superior performance over existing MIL- and VLM-based methods in few-shot WSI classification tasks. Our code is publicly available at https://github.com/LTS5/SLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。