用混合架构让3D医学影像分割更高效,参数少75倍仍保持高精度。
SegMaFormer: A Hybrid State-Space and Transformer Model for Efficient Segmentation
- 早期用Mamba捕获空间上下文,后期用Transformer细化特征。
- 在三个数据集上达到顶尖模型的分割精度,参数减少75倍。
- 适合资源有限但需高精度分割的医疗图像研究者。
Transformer与基于Mamba的架构显著推进了3D医学图像分割,实现全局上下文建模,这是传统卷积网络难以做到的。然而,最先进的Transformer模型通常计算复杂度高、参数量大,这对体数据尤其不利,且受限于标注医学影像数据集的稀缺性。为此,本文提出SegMaFormer,一种轻量级混合架构,在分层体编码器中融合Mamba与Transformer模块,实现高效的长距离依赖建模。模型在早期高分辨率阶段采用Mamba层以降低计算开销并捕捉关键空间上下文,后期低分辨率阶段保留自注意力机制以优化特征表示。该设计结合广义旋转位置编码,增强空间感知能力。尽管结构紧凑,SegMaFormer在三个公开基准(Synapse、BraTS、ACDC)上表现优异,其Dice系数媲美远大于它的模型。实证表明,本方法相比当前最优模型,参数量减少高达75倍,显著降低浮点运算次数(FLOPs),为3D医学图像分割提供高效高性能解决方案。
原文摘要 · Abstract (English)
The advent of Transformer and Mamba-based architectures has significantly advanced 3D medical image segmentation by enabling global contextual modeling, a capability traditionally limited in Convolutional Neural Networks (CNNs). However, state-of-the-art Transformer models often entail substantial computational complexity and parameter counts, which is particularly prohibitive for volumetric data and further exacerbated by the limited availability of annotated medical imaging datasets. To address these limitations, this work introduces SegMaFormer, a lightweight hybrid architecture that synergizes Mamba and Transformer modules within a hierarchical volumetric encoder for efficient long-range dependency modeling. The model strategically employs Mamba-based layers in early, high-resolution stages to reduce computational overhead while capturing essential spatial context, and reserves self-attention mechanisms for later, lower-resolution stages to refine feature representation. This design is augmented with generalized rotary position embeddings to enhance spatial awareness. Despite its compact structure, SegMaFormer achieves competitive performance on three public benchmarks (Synapse, BraTS, and ACDC), matching the Dice coefficient of significantly larger models. Empirically, our approach reduces parameters by up to 75x and substantially decreases FLOPs compared to current state-of-the-art models, establishing an efficient and high-performing solution for 3D medical image segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。