基于体积金字塔Transformer的多模态心脏图像分割新方法
MCSeg: Pre-training and Fine-tuning Volumetric Pyramid Transformer for Multi-modal Cardiac Image Segmentation
- 提出SFP结构，打通ViT与CNN的特征层级鸿沟
- 在4个数据集上超越11种先进方法，少样本下表现突出
- 适合医学图像分割研究者，尤其关注少样本场景
自动心脏图像分割对心脏病诊断与治疗至关重要。本文提出MCSeg，一种面向多模态心脏分割的体积化Transformer网络。为克服现有混合网络的架构不匹配问题，提出新型缩放特征金字塔（SFP），通过将单尺度3D ViT编码器输出转换为分层特征金字塔，有效连接3D ViT编码器与多尺度CNN解码器，确保全局上下文信息被充分利用。训练采用自监督预训练+微调范式：先通过掩码图像建模预训练ViT编码器，再在下游任务中引入区域互信息（RMI）损失以提升边界分割精度。实验表明，MCSeg在CT数据集ImageCHD、多模态数据集MM-WHS、MRI数据集HVSMR-2.0和MSD Heart上均持续优于11种最先进方法，且在少样本实验中表现优异，展现出强泛化能力。代码与预训练的ViT-B权重已开源。
原文摘要 · Abstract (English)
Automatic cardiac image segmentation is pivotal for diagnosing and treating cardiac diseases. In this work, we introduce MCSeg, a volumetric transformer-based network tailored for multi-modal cardiac segmentation. To overcome the architectural mismatch inherent in existing hybrid networks, we propose a novel Scaling Feature Pyramid (SFP). Unlike conventional skip connections, the SFP effectively bridges the single-scale 3D Vision Transformer (ViT) encoder and the multi-scale CNN decoder by transforming the ViT's output into a hierarchical feature pyramid, ensuring that global contextual information is effectively leveraged. For the training paradigm, the ViT encoder first undergoes self-supervised pre-training via masked image modeling. Subsequently, the network is fine-tuned on downstream tasks, during which a regional mutual information (RMI) loss is integrated to improve boundary segmentation accuracy. In experiments, MCSeg consistently outperforms eleven SOTA methods on CT dataset ImageCHD, multi-modal dataset MM-WHS, MRI dataset HVSMR-2.0 and MSD Heart, highlighting the effectiveness of our MCSeg for multi-modal cardiac segmentation tasks. Furthermore, MCSeg's superior performance in few-shot experiment showcases its significant potential in adapting to limited data scenarios. Codes and pre-trained ViT-B weights are open-sourced at https://openi.pcl.ac.cn/OpenMedIA/MCSeg
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。