一个模型搞定多种癌症病理图像分类,还能自动生成诊断结果。
MECFormer: Multi-task Whole Slide Image Classification with Expert Consultation Network
- 用专家咨询网络提升多任务学习能力,专注每个任务细节。
- 在5个数据集上表现优于现有单任务模型,涵盖4种器官和5类癌症任务。
- 引入自回归解码器实现灵活分类输出,适合临床多病种诊断场景。
全切片图像(WSI)分类对癌症临床诊断至关重要。一张WSI可达千兆像素级,通常被划分为多个图像块,通过多实例学习(MIL)模型处理。以往的MIL模型仅在特定器官的单一任务上评估,未探索单模型处理多任务的能力。本文提出MECFormer,一种基于生成式Transformer的模型,可统一处理多项任务。为增强多任务协同学习并聚焦各任务,我们在Transformer前端引入专家咨询网络(投影层)。此外,通过语言解码器实现自回归解码,支持灵活分类输出。在包含四种器官、一项癌症分类与四项癌亚型分类任务的五个数据集上进行大量实验,MECFormer性能显著优于当前最优的独立MIL模型。
原文摘要 · Abstract (English)
Whole slide image (WSI) classification is a crucial problem for cancer diagnostics in clinics and hospitals. A WSI, acquired at gigapixel size, is commonly tiled into patches and processed by multiple-instance learning (MIL) models. Previous MIL-based models designed for this problem have only been evaluated on individual tasks for specific organs, and the ability to handle multiple tasks within a single model has not been investigated. In this study, we propose MECFormer, a generative Transformer-based model designed to handle multiple tasks within one model. To leverage the power of learning multiple tasks simultaneously and to enhance the model's effectiveness in focusing on each individual task, we introduce an Expert Consultation Network, a projection layer placed at the beginning of the Transformer-based model. Additionally, to enable flexible classification, autoregressive decoding is incorporated by a language decoder for WSI classification. Through extensive experiments on five datasets involving four different organs, one cancer classification task, and four cancer subtyping tasks, MECFormer demonstrates superior performance compared to individual state-of-the-art multiple-instance learning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。