提出多倍率融合方法,提升病理图像区域表征的泛化能力。
Mixed Magnification Aggregation for Generalizable Region-Level Representations in Computational Pathology
- 用多倍率切片联合训练区域聚合编码器,融合不同尺度特征。
- 在多种癌症生物标志物预测任务中实现依赖癌种的性能提升。
- 适合需要多尺度分析的病理图像建模研究者使用。
近年来,计算病理学标准流程是将全幻灯片图像切分为小块,用基础模型处理这些小块,再基于所得表征构建特定任务模型。已有至少15种基础模型被提出,大多数仅在20×倍率下使用小块进行训练。然而,某些组织学特征需更大上下文窗口才能辨识,病理科医生常需放大缩小观察幻灯片。此外,以20×倍率生成224×224像素的小块会导致每张幻灯片产生大量小块,总尺寸可达吉字节级别。为更准确捕捉多分辨率特征并探索减少每张幻灯片表征数量的可能性,我们提出一种区域级混合编码器。该方法通过掩码嵌入建模预训练步骤,联合融合多倍率基础模型的小块表征。我们探索了该混合倍率区域聚合器的预训练设计空间,并在转移至多种癌症类型的生物标志物预测任务上评估模型表现。结果表明,在不同癌症类型中均实现预测性能提升,凸显空间上下文与理解的重要性。
原文摘要 · Abstract (English)
In recent years, a standard computational pathology workflow has emerged where whole slide images are cropped into tiles, these tiles are processed using a foundation model, and task-specific models are built using the resulting representations. At least 15 different foundation models have been proposed, and the vast majority are trained exclusively with tiles using the 20$\times$ magnification. However, it is well known that certain histologic features can only be discerned with larger context windows and requires a pathologist to zoom in and out when analyzing a whole slide image. Furthermore, creating 224$\times$224 pixel crops at 20$\times$ leads to a large number of tiles per slide, which can be gigapixel in size. To more accurately capture multi-resolution features and investigate the possibility of reducing the number of representations per slide, we propose a region-level mixing encoder. Our approach jointly fuses image tile representations of a mixed magnification foundation model using a masked embedding modeling pretraining step. We explore a design space for pretraining the proposed mixed-magnification region aggregators and evaluate our models on transfer to biomarker prediction tasks representing various cancer types. Results demonstrate cancer dependent improvements in predictive performance, highlighting the importance of spatial context and understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。