用语言模型指导分步选图,让病理切片分析快3倍还更准。
LanGuSTE: Language-Guided Coarse-to-Fine Patch Selection for Efficient Whole Slide Image Analysis
- 先用低分辨率图匹配高分辨率图,再用语言模型筛选关键区域。
- 处理时间降为原来的1/3,诊断准确率反而更高。
- 适合做大规模病理图像分析的研究者和临床医生。
病理学中的全切片图像(WSI)因达十亿像素级而计算成本极高,通常每张切片需处理数万至数十万张高分辨率图像块。传统流程中,全面处理所有图像块使预处理耗时远超下游模型训练。现有图像块选择方法存在根本矛盾:所有图像块仍需在训练阶段甚至推理阶段提取并编码后才能剔除无关块。为此,我们提出LanGuSTE,一种融合病理领域视觉-语言模型(VLM)与大语言模型(LLM)知识的高效图像块选择框架,包含两个核心模块:跨尺度视觉提示调优(CS-VPT)和粗到细图像块选择。CS-VPT通过对比学习将低分辨率图像块与其空间对应高分辨率图像块对齐,将细微诊断语义传递至低分辨率表示。图像块选择模块则利用VLM表征和LLM生成的病理解剖特异性描述,在粗到细层级识别信息丰富区域,仅编码对应高分辨率图像块以降低预处理时间。大量实验表明,LanGuSTE将整体WSI处理时间减少约3倍,同时诊断性能达到或超过全面图像块处理及近期最先进选择方法。
原文摘要 · Abstract (English)
Whole slide images (WSIs) in computational pathology pose a major computational challenge due to their gigapixel scale, often requiring tens to hundreds of thousands of high-resolution patches to be processed per slide. In conventional WSI pipelines, exhaustive high-resolution patch processing makes preprocessing far more time-consuming than downstream model training. Existing patch selection methods suffer from a fundamental paradox: all patches must still be extracted and encoded at least during training, and sometimes during both training and inference, before irrelevant ones can be discarded. To address this, we propose LanGuSTE, an efficient patch selection framework that integrates pathology-domain vision-language models (VLMs) and knowledge derived from large language models (LLMs) through two key modules: Cross- Scale Visual Prompt Tuning (CS-VPT) and coarse-to-fine patch selection. CS-VPT aligns low-resolution patches with their spatially corresponding high-resolution patches through contrastive learning, transferring fine-grained diagnostic semantics into low-resolution representations. The patch selection module then leverages VLM representations and LLM-generated pathology-specific descriptions to identify informative regions in a coarse-to-fine manner, encoding only the corresponding high-resolution patches to reduce preprocessing time. Extensive experiments demonstrate that LanGuSTE reduces overall WSI processing time to approximately 3x while achieving diagnostic performance comparable to or better than exhaustive patch processing and recent state-of-the-art patch-selection methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。