用视觉语言对齐实现病理切片精准检索,提升诊断一致性。
Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment
- 通过注意力拼贴+跨模态对比学习,融合细粒度形态与高层语义。
- 在6个器官、4个数据集上优于传统方法,多中心研究证实诊断准确率提升。
- 适合临床辅助诊断、教学案例查找及跨机构病理数据共享。
组织学切片的快速数字化为临床与科研工作中的计算工具开辟了新可能。其中,基于内容的切片检索尤为突出,可帮助病理科医生识别形态和语义相似病例,支持精准诊断、提高观察者间一致性,并辅助实例化教学。然而,全切片图像(WSIs)的吉字节级规模以及大量无关内容中捕捉细微语义差异的困难,使有效检索仍具挑战。为此,我们提出PathSearch框架,通过视觉-语言对比学习统一细粒度注意力拼贴表示与全局切片嵌入,实现精准且灵活的检索。该框架在6,926对切片-报告语料上训练,能同时捕捉细粒度形态特征与高层语义模式。支持两种功能:(1) 基于拼贴的图像到图像检索,确保高效准确;(2) 多模态检索,允许文本查询直接匹配相关切片。PathSearch在四个公开病理数据集和三个院内队列上进行了严格评估,涵盖解剖部位检索、肿瘤亚型分类、良恶性区分及多种器官(乳腺、肺、肾、肝、胃)的分级任务。外部结果表明,PathSearch优于传统图像到图像检索框架。多中心阅片研究进一步证明,其显著提升诊断准确性、增强诊断信心并改善病理科医生间的观察一致性。这些结果确立了PathSearch作为数字病理领域可扩展、通用的检索解决方案。
原文摘要 · Abstract (English)
The rapid digitization of histopathology slides has opened up new possibilities for computational tools in clinical and research workflows. Among these, content-based slide retrieval stands out, enabling pathologists to identify morphologically and semantically similar cases, thereby supporting precise diagnoses, enhancing consistency across observers, and assisting example-based education. However, effective retrieval of whole slide images (WSIs) remains challenging due to their gigapixel scale and the difficulty of capturing subtle semantic differences amid abundant irrelevant content. To overcome these challenges, we present PathSearch, a retrieval framework that unifies fine-grained attentive mosaic representations with global-wise slide embeddings aligned through vision-language contrastive learning. Trained on a corpus of 6,926 slide-report pairs, PathSearch captures both fine-grained morphological cues and high-level semantic patterns to enable accurate and flexible retrieval. The framework supports two key functionalities: (1) mosaic-based image-to-image retrieval, ensuring accurate and efficient slide research; and (2) multi-modal retrieval, where text queries can directly retrieve relevant slides. PathSearch was rigorously evaluated on four public pathology datasets and three in-house cohorts, covering tasks including anatomical site retrieval, tumor subtyping, tumor vs. non-tumor discrimination, and grading across diverse organs such as breast, lung, kidney, liver, and stomach. External results show that PathSearch outperforms traditional image-to-image retrieval frameworks. A multi-center reader study further demonstrates that PathSearch improves diagnostic accuracy, boosts confidence, and enhances inter-observer agreement among pathologists in real clinical scenarios. These results establish PathSearch as a scalable and generalizable retrieval solution for digital pathology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。