用病理大模型做有丝分裂细胞检测,效果媲美专用模型。
Beyond Classification: Pathology Foundation Models as Detection Encoders for Mitotic Figures

- 用大模型提取特征,直接接入检测框架
- H-optimus-0和Virchow在多数据集上表现接近专用模型
- 适合想省去训练成本的病理图像检测研究者
病理基础模型(FMs)在大量无标注数据上训练,生成的隐空间可用于下游分类任务。但当前尚不清楚这些模型的隐空间是否具备足够判别力和空间分辨率,可直接用于密集目标检测。本文评估了多种主流病理大模型(UNI、UNI2-h、Virchow、Virchow2、H-optimus-0、H-optimus-1)在有丝分裂细胞检测中的表现,对比基于ResNet50的端到端模型。将这些模型作为骨干网络,结合RetinaNet、Faster R-CNN、Deformable DETR三类检测器,在多领域MIDOG++数据集和域外测试集TUPAC16上进行实验。结果表明,H-optimus-0与Virchow模型表现优异,说明当前基于图像级自监督训练的大模型隐空间已适用于有丝分裂细胞检测,且在域外测试中略具鲁棒性。所有代码公开于https://github.com/DeepMicroscopy/FM4MFdet。
原文摘要 · Abstract (English)
Pathology foundation models (FMs) are models trained on vast amounts of typically unlabeled data and have been shown to yield regularized latent spaces that can be used effectively in downstream classification tasks. This is also true for the classification of mitotic figures vs. other cells. However, it is so far unclear if the latent space of current FMs provides features that are discriminant and spatially suitably resolved to also serve as a backbone for dense object detection paradigms. In this work, we investigate this question for common current pathology FMs (UNI, UNI2-h, Virchow, Virchow2, H-optimus-0, H-optimus-1) and compare their performance against a fully end-to-end trained baseline based on a ResNet50 architecture. We combine FM backbones with representatives of single stage, dual stage and self-attention-based detectors (RetinaNet, Faster R-CNN, Deformable DETR respectively) on the multi-domain MIDOG++ dataset, and on the TUPAC16 dataset as an out-of-domain case. We show that the H-optimus-0 and Virchow models yielded competitive performance, indicating that the latent spaces of current FMs, all trained on image-level self-supervision, are suitable for direct mitotic figure detection and may be slightly more robust on our out-of-domain test case. All code is made available publicly at https://github.com/DeepMicroscopy/FM4MFdet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。