对比10个基础模型在病理图像分割中的表现,发现多模态模型更优。
Benchmarking Computational Pathology Foundation Models For Semantic Segmentation
- 用注意力图生成像素特征,结合XGBoost快速评估,无需微调。
- CONCH在4个数据集上表现最佳,融合三模型可提升7.95%精度。
- 不同训练数据的模型互补,组合使用效果更佳,适合病理分析研究者。
近年来,如CLIP、DINO和CONCH等基础模型在多种成像任务中展现出出色的领域泛化与无监督特征提取能力。然而,针对组织病理学中像素级语义分割的系统性独立评估仍较为缺乏。本研究提出一种稳健的基准测试方法,评估10个基础模型在四个组织病理学数据集上的表现,涵盖形态组织区域与细胞/核分割任务。方法利用基础模型的注意力图作为像素级特征,通过XGBoost分类器进行快速、可解释且模型无关的评估,无需微调。结果表明,视觉-语言基础模型CONCH在多个数据集上表现最优,PathDino紧随其后。进一步分析显示,基于不同病理队列训练的模型捕获了互补的形态表征,特征拼接后分割性能显著提升。将CONCH、PathDino与CellViT特征融合,在所有数据集上平均提升7.95%性能,表明基础模型集成能更好泛化于多样化的病理分割任务。
原文摘要 · Abstract (English)
In recent years, foundation models such as CLIP, DINO,and CONCH have demonstrated remarkable domain generalization and unsupervised feature extraction capabilities across diverse imaging tasks. However, systematic and independent evaluations of these models for pixel-level semantic segmentation in histopathology remain scarce. In this study, we propose a robust benchmarking approach to asses 10 foundational models on four histopathological datasets covering both morphological tissue-region and cellular/nuclear segmentation tasks. Our method leverages attention maps of foundation models as pixel-wise features, which are then classified using a machine learning algorithm, XGBoost, enabling fast, interpretable, and model-agnostic evaluation without finetuning. We show that the vision language foundation model, CONCH performed the best across datasets when compared to vision-only foundation models, with PathDino as close second. Further analysis shows that models trained on distinct histopathology cohorts capture complementary morphological representations, and concatenating their features yields superior segmentation performance. Concatenating features from CONCH, PathDino and CellViT outperformed individual models across all the datasets by 7.95% (averaged across the datasets), suggesting that ensembles of foundation models can better generalize to diverse histopathological segmentation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。