arXiv:2502.02471cs.CVcs.AI2025-02

对比通用与病理专用模型的图像块嵌入能力,发现后者在细胞分割分类中更优。

Mind the Gap: Evaluating Patch Embeddings from General-Purpose and Histopathology Foundation Models for Cell Segmentation and Classification

  • 用统一解码器比较不同编码器的嵌入效果,冻结参数评估预训练特征。
  • 在PanNuke、CoNIC和新数据集上,病理模型在细胞分割与分类上表现更好。
  • 适合从事病理分析、脑细胞结构研究的科研人员参考模型选型。

近期基础模型推动了计算机视觉发展,但在数字病理领域,特定于病理的模型相较于通用模型在细胞分析任务中的优势仍不明确。本研究通过分析多层级图像块嵌入,评估通用模型(ImageNet-22K、LVD-142M预训练)与最新病理基础模型(UNI、Virchow2、Prov-GigaPath)在细胞实例分割与分类中的表现。采用统一解码器架构,通过跳跃连接融合不同深度编码器的嵌入,生成语义图与距离图,并后处理生成细胞实例掩码及类型分类。所有编码器保持冻结以评估其预训练特征提取能力。在PanNuke、CoNIC以及新提出的尼氏染色CytoDArk0数据集上进行评估,结果揭示了两类模型在细胞级检测、分割精度与分类性能上的差异,为细胞导向的病理学与脑细胞架构分析提供了模型选择依据。

原文摘要 · Abstract (English)

Recent advancements in foundation models have transformed computer vision, driving significant performance improvements across diverse domains, including digital histopathology. However, the advantages of domain-specific histopathology foundation models over general-purpose models for specialized tasks such as cell analysis remain underexplored. This study investigates the representation learning gap between these two categories by analyzing multi-level patch embeddings applied to cell instance segmentation and classification. We implement an encoder-decoder architecture with a consistent decoder and various encoders. These include convolutional, vision transformer (ViT), and hybrid encoders pre-trained on ImageNet-22K or LVD-142M, representing general-purpose foundation models. These are compared against ViT encoders from the recently released UNI, Virchow2, and Prov-GigaPath foundation models, trained on patches extracted from hundreds of thousands of histopathology whole-slide images. The decoder integrates patch embeddings from different encoder depths via skip connections to generate semantic and distance maps. These maps are then post-processed to create instance segmentation masks where each label corresponds to an individual cell and to perform cell-type classification. All encoders remain frozen during training to assess their pre-trained feature extraction capabilities. Using the PanNuke and CoNIC histopathology datasets, and the newly introduced Nissl-stained CytoDArk0 dataset for brain cytoarchitecture studies, we evaluate instance-level detection, segmentation accuracy, and cell-type classification. This study provides insights into the comparative strengths and limitations of general-purpose vs. histopathology foundation models, offering guidance for model selection in cell-focused histopathology and brain cytoarchitecture analysis workflows.

病理图像细胞分割基础模型图像嵌入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。