用基础模型实现高效自适应细胞分割分类,零样本表现佳。
CellViT++: Energy-Efficient and Adaptive Cell Segmentation and Classification Using Foundation Models
- 用视觉变换器+基础模型同时生成深度特征与分割图
- 仅需少量数据即可实现零样本分割与高精度分类
- 适合病理医生与科研人员快速部署,支持无标注数据生成
数字病理学是疾病诊断与治疗的核心。关键任务是在苏木精-伊红染色图像中识别和分割细胞。现有方法通常需要大量标注数据训练,且局限于预定义的细胞分类体系。为此,我们提出 $ ext{CellViT}^{ ext{++}}$,一种通用细胞分割框架。该框架利用带有基础模型的视觉变换器作为编码器,同时计算深层细胞特征与分割掩码。为适应未见细胞类型,采用计算高效的策略,仅需极少数据训练,显著降低碳足迹。在七个不同数据集上验证,覆盖广泛细胞类型、器官及临床场景,均取得优异性能。框架实现卓越的零样本分割与数据高效分类。此外,展示其可借助免疫荧光染色生成训练数据,无需病理医生标注。自动化数据生成方法优于人工标注训练网络,证明其在无专家标注下构建高质量数据集的有效性。为推动数字病理发展,$ ext{CellViT}^{ ext{++}}$ 已开源,提供用户友好的网页可视化与标注界面,代码见 https://github.com/TIO-IKIM/CellViT-plus-plus。
原文摘要 · Abstract (English)
Digital Pathology is a cornerstone in the diagnosis and treatment of diseases. A key task in this field is the identification and segmentation of cells in hematoxylin and eosin-stained images. Existing methods for cell segmentation often require extensive annotated datasets for training and are limited to a predefined cell classification scheme. To overcome these limitations, we propose $\text{CellViT}^{\scriptscriptstyle ++}$, a framework for generalized cell segmentation in digital pathology. $\text{CellViT}^{\scriptscriptstyle ++}$ utilizes Vision Transformers with foundation models as encoders to compute deep cell features and segmentation masks simultaneously. To adapt to unseen cell types, we rely on a computationally efficient approach. It requires minimal data for training and leads to a drastically reduced carbon footprint. We demonstrate excellent performance on seven different datasets, covering a broad spectrum of cell types, organs, and clinical settings. The framework achieves remarkable zero-shot segmentation and data-efficient cell-type classification. Furthermore, we show that $\text{CellViT}^{\scriptscriptstyle ++}$ can leverage immunofluorescence stainings to generate training datasets without the need for pathologist annotations. The automated dataset generation approach surpasses the performance of networks trained on manually labeled data, demonstrating its effectiveness in creating high-quality training datasets without expert annotations. To advance digital pathology, $\text{CellViT}^{\scriptscriptstyle ++}$ is available as an open-source framework featuring a user-friendly, web-based interface for visualization and annotation. The code is available under https://github.com/TIO-IKIM/CellViT-plus-plus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。