用自然语言实现病理图像分割,免去手动标注点框。
Segment Anything in Pathology Images with Natural Language
- 通过自然语言提示进行病理图像语义分割,无需点或框输入。
- 在275k数据上训练,整体Dice得分领先现有模型0.429。
- 适合病理医生和AI研究者用于可解释性癌症诊断支持。
病理图像分割对癌症诊断与预后分析至关重要,但受限于标注数据少和类别定义僵化。为此,我们提出PathSegmentor——首个专为病理图像设计的文本提示分割基础模型,并构建了包含275,000张图像-掩码-标签三元组、覆盖160个多样类别的最大最全面病理分割数据集PathSeg,源自21个公开来源。用户仅需自然语言提示即可完成语义分割,无需繁琐的空间输入。实验表明,PathSegmentor在准确率和泛化能力上均优于专用模型,整体Dice分数分别比现有空间提示与文本提示模型高出0.145和0.429,对复杂结构分割表现稳健,且能在外部数据集上良好迁移。其输出还可用于特征重要性估计与影像生物标志物发现,提升诊断模型可解释性,为精准肿瘤学中的可解释AI发展提供支持。
原文摘要 · Abstract (English)
Pathology image segmentation is crucial in computational pathology for analyzing histological features relevant to cancer diagnosis and prognosis. However, current methods face major challenges in clinical applications due to limited annotated data and restricted category definitions. To address these limitations, we propose PathSegmentor, the first text-prompted segmentation foundation model designed specifically for pathology images. We also introduce PathSeg, the largest and most comprehensive dataset for pathology segmentation, built from 21 public sources and containing 275k image-mask-label triples across 160 diverse categories. With PathSegmentor, users can perform semantic segmentation using natural language prompts, eliminating the need for laborious spatial inputs such as points or boxes. Extensive experiments demonstrate that PathSegmentor outperforms specialized models with higher accuracy and broader applicability, while maintaining a compact architecture. It significantly surpasses existing spatial- and text-prompted models by 0.145 and 0.429 in overall Dice scores, respectively, showing strong robustness in segmenting complex structures and generalizing to external datasets. Moreover, PathSegmentor's outputs enhance the interpretability of diagnostic models through feature importance estimation and imaging biomarker discovery, offering pathologists evidence-based support for clinical decision-making. This work advances the development of explainable AI in precision oncology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。