arXiv:2606.16658cs.CV2026-06

用视觉语言模型做病理图像分割,无需标注就能跨染色类型通用。

Vision-Language Models as Zero-Annotation Oracles in Histopathology

  • 把组织分割当作图像识别问题,用通用VLM当零标注裁判。
  • 在非标准染色上Dice达0.858(琼斯银染)和0.853(弹性纤维染色),方差降低7倍。
  • 适合处理新染色、小样本或资源受限场景的病理分析。

前景分割是计算病理学流程的关键第一步,但现有方法依赖手工调参或受特定染色与扫描仪限制的监督模型,在琼斯银染、弹性纤维-范吉森染色等特殊染色下会无声失效。本文提出一种由粗到精的方法,将前景分割重构为视觉感知任务,利用通用视觉语言模型(VLMs)作为零标注裁判。核心洞察是组织与背景区分本质上是自然图像识别问题,而非病理专属任务,因此训练于互联网规模数据的VLM具备更强泛化能力。我们构建了包含75张肾移植全幻灯片的Leica-75基准数据集,涵盖三种染色类别。在该数据集上,本方法在分布外染色上的分割质量最优(琼斯染色Dice 0.858±0.027,EVG染色Dice 0.853±0.041),且跨染色方差比最佳监督基线低7倍,同时在分布内H&E染色上保持竞争力。少量提示(few-shot prompting)结合自动精选样例(Auto-context)可有效解决困难案例(Stress-32子集,n=32,2B模型的Dice在0.470至0.819之间)。基于VLM的标注审核结果接近人类专家共识(模糊检测κ=0.989;分割掩码评分准确率均值0.708,人类为0.646)。由此生成的伪标签用于蒸馏轻量级学生模型,性能媲美教师模型,运行成本仅为一小部分。本框架为数字病理中的持续基础设施瓶颈提供了一种系统性、可扩展的解决方案。

原文摘要 · Abstract (English)

Foreground segmentation is the critical first step of every computational pathology pipeline, yet existing methods rely on hand-tuned heuristics or supervised models that overfit to narrow stain and scanner distributions, failing silently on specialised stains such as Jones silver or Elastica van Gieson. We propose a coarse-to-fine approach that recasts foreground segmentation as a visual perception task and leverages general-purpose vision-language models (VLMs) as zero-annotation oracles. Our key insight is that tissue-versus-background discrimination is a natural-image recognition problem, not a histopathological one, so VLMs trained on internet-scale corpora generalise where domain-specific models cannot. We introduce Leica-75, a benchmark of 75 renal transplant whole-slide images spanning three stain families. On Leica-75, our method achieves the highest segmentation quality on out-of-distribution stains (Dice 0.858 +/- 0.027 on Jones, 0.853 +/- 0.041 on EVG) with 7x lower cross-stain variance than the best supervised baseline, while remaining competitive on in-distribution H&E. Few-shot prompting with automatically curated exemplars (Auto-context) rescues hard cases on Stress-32 (n=32), a curated stress-test subset (Dice 0.470 to 0.819 for the 2B model). VLM-based annotation review matches human expert consensus (kappa=0.989 for blur detection; mean precision/recall grading accuracy 0.708 vs. human 0.646 for segmentation mask review). The resulting pseudo-labels are used to distil lightweight student models that are as performant as the teacher model while running for a fraction of the cost. Our framework provides a principled, scalable solution to a persistent infrastructure bottleneck in digital pathology.

病理图像视觉语言模型零样本自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。