PRISM2用临床对话训练,让病理模型能理解全切片并支持诊断问答。
PRISM2: Unlocking Multi-Modal General Pathology AI with Clinical Dialogue
- 基于70万份病理报告对,用临床对话监督多模态预训练
- 无需微调即可达到临床级癌症检测性能,且在多个任务上表现顶尖
- 适合需要通用病理分析能力的研究者与临床辅助系统开发者
计算病理学的快速发展得益于基础模型的兴起。这些模型正从图像块编码转向全切片理解,但其临床实用性仍受限。本文提出PRISM2,一个在70万份诊断样本-报告对上训练的多模态全切片基础模型,数据规模为迄今最大:视觉部分包含230万张全切片图像,语言部分包含1400万组问答对。通过临床对话监督学习,PRISM2将组织形态特征与诊断推理语言对齐,生成支持直接问答和下游任务迁移的切片级表征。未经额外训练,其癌症检测性能即达到或超越临床级产品水平,且在其他任务中保持领先。以生存预测为例,使用大规模数据进行特定任务微调,可超越专用模型,进一步提升性能。结果表明,语言监督的预训练为学习可泛化的病理表征提供了可扩展、具临床意义的信号,弥合了人类诊断推理与基础模型性能之间的差距。
原文摘要 · Abstract (English)
Recent rapid progress in the field of computational pathology has been enabled by foundation models. These models are beginning to move beyond encoding image patches towards whole-slide understanding but their clinical utility remains limited. In this work, we present PRISM2, a multimodal slide-level foundation model trained on data from 700,000 diagnostic specimen-report pairs, the largest vision (2.3 million whole slide images) and language (14M question-answer pairs) histopathology dataset to date. By learning through clinical-dialogue supervision, PRISM2 aligns histomorphologic features with the language of diagnostic reasoning, producing slide-level representations that support both direct diagnostic question-answering and transferable embeddings for downstream tasks. Without additional training, PRISM2 matches or exceeds the cancer-detection performance of clinical-grade products. This is observed without loss of generality on other tasks, where PRISM2 achieves top performance. Finally, using survival prediction as the example, we show that task-specific finetuning with a large dataset can outperform task-specific models, further improving performance. These results demonstrate how language-supervised pretraining provides a scalable, clinically grounded signal for learning generalizable pathology representations, bridging human diagnostic reasoning and foundation-model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。