arXiv:2501.17822eess.IVcs.AI2025-01

比较多种方法将病理切片拼图嵌入合并为单一向量的性能。

Aggregation Schemes for Single-Vector WSI Representation Learning in Digital Pathology

  • 用多种集合学习技术融合病理切片的图像块嵌入。
  • 在膀胱、乳腺等四个癌种数据集上评估检索效果,验证方法优劣。
  • 对比传统非聚合方法,发现特定聚合策略显著提升检索精度。

在数字病理学中,高效整合全切片图像(WSI)的关键步骤是为每张WSI分配一个高质量的单一特征向量(即嵌入)。尽管预训练深度神经网络可轻松提取子图像(如瓦片或补丁)的嵌入,但因WSI分辨率极高、达千兆像素级,无法直接输入现有GPU作为单张图像处理。因此,通常将WSI拆分为多个补丁,并分别输入预训练模型,得到一组补丁嵌入。此时,WSI表示学习转化为集合表示学习问题:对每个WSI,获得一组补丁嵌入。本文评估了近年来多种集合学习技术在四类主要癌症部位(膀胱、乳腺、肾、结肠)的TCGA数据集上的WSI搜索性能,包括简单平均/最大池化、Deep Sets、记忆网络、焦点注意力、高斯混合模型Fisher向量以及深度稀疏与二值化Fisher向量。同时,将这些方法的检索性能与不进行聚合的最小距离中位数方法进行基准对比。

原文摘要 · Abstract (English)

A crucial step to efficiently integrate Whole Slide Images (WSIs) in computational pathology is assigning a single high-quality feature vector, i.e., one embedding, to each WSI. With the existence of many pre-trained deep neural networks and the emergence of foundation models, extracting embeddings for sub-images (i.e., tiles or patches) is straightforward. However, for WSIs, given their high resolution and gigapixel nature, inputting them into existing GPUs as a single image is not feasible. As a result, WSIs are usually split into many patches. Feeding each patch to a pre-trained model, each WSI can then be represented by a set of patches, hence, a set of embeddings. Hence, in such a setup, WSI representation learning reduces to set representation learning where for each WSI we have access to a set of patch embeddings. To obtain a single embedding from a set of patch embeddings for each WSI, multiple set-based learning schemes have been proposed in the literature. In this paper, we evaluate the WSI search performance of multiple recently developed aggregation techniques (mainly set representation learning techniques) including simple average or max pooling operations, Deep Sets, Memory networks, Focal attention, Gaussian Mixture Model (GMM) Fisher Vector, and deep sparse and binary Fisher Vector on four different primary sites including bladder, breast, kidney, and Colon from TCGA. Further, we benchmark the search performance of these methods against the median of minimum distances of patch embeddings, a non-aggregating approach used for WSI retrieval.

病理图像嵌入聚合集合学习医学影像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。