arXiv:2602.00650cs.CV2026-02

将SAM与Mamba结合,高效实现3D医学图像分割。

A Hybrid Mamba-SAM Architecture for Efficient 3D Medical Image Segmentation

  • 用冻结SAM+可训练VMamba双分支融合特征,提升医学图像适应性。
  • 在ACDC数据集上,心肌和左心室分割Dice达0.910和0.971。
  • 推理速度达4.77 FPS,适合临床实时应用。

准确分割3D医学影像(如MRI、CT)对临床诊断与治疗规划至关重要。通用基础模型如分割一切模型(SAM)虽具强大表征能力,但受限于领域偏移、固有2D设计及微调高计算成本,在医学影像中表现不佳。为此,我们提出Mamba-SAM,一种新颖高效的混合架构,融合冻结的SAM编码器与具备线性时间效率和长程建模能力的Mamba状态空间模型(SSMs)。探索两种参数高效适配策略:其一为双分支结构,通过交叉注意力显式融合来自冻结SAM编码器的通用特征与可训练VMamba编码器学习的领域特定表示;其二为适配器方法,将轻量级3D感知的三平面Mamba(TPMamba)模块注入冻结的SAM ViT编码器,隐式建模体积分量上下文。在此框架中引入多频带门控卷积(MFGC),通过3D离散余弦变换与自适应门控联合分析空间与频域信息,增强特征表达。在ACDC心脏MRI数据集上的大量实验表明,双分支Mamba-SAM-Base模型平均Dice得分为0.906,接近UNet++(0.907),并在心肌(0.910)与左心室(0.971)分割上超越所有基线。适配器型TP MFGC变体实现更高推理速度(4.77 FPS)与良好准确率(0.880 Dice)。结果表明,将基础模型与高效SSM架构混合,是3D医学图像分割的实用有效方案。

原文摘要 · Abstract (English)

Accurate segmentation of 3D medical images such as MRI and CT is essential for clinical diagnosis and treatment planning. Foundation models like the Segment Anything Model (SAM) provide powerful general-purpose representations but struggle in medical imaging due to domain shift, their inherently 2D design, and the high computational cost of fine-tuning. To address these challenges, we propose Mamba-SAM, a novel and efficient hybrid architecture that combines a frozen SAM encoder with the linear-time efficiency and long-range modeling capabilities of Mamba-based State Space Models (SSMs). We investigate two parameter-efficient adaptation strategies. The first is a dual-branch architecture that explicitly fuses general features from a frozen SAM encoder with domain-specific representations learned by a trainable VMamba encoder using cross-attention. The second is an adapter-based approach that injects lightweight, 3D-aware Tri-Plane Mamba (TPMamba) modules into the frozen SAM ViT encoder to implicitly model volumetric context. Within this framework, we introduce Multi-Frequency Gated Convolution (MFGC), which enhances feature representation by jointly analyzing spatial and frequency-domain information via 3D discrete cosine transforms and adaptive gating. Extensive experiments on the ACDC cardiac MRI dataset demonstrate the effectiveness of the proposed methods. The dual-branch Mamba-SAM-Base model achieves a mean Dice score of 0.906, comparable to UNet++ (0.907), while outperforming all baselines on Myocardium (0.910) and Left Ventricle (0.971) segmentation. The adapter-based TP MFGC variant offers superior inference speed (4.77 FPS) with strong accuracy (0.880 Dice). These results show that hybridizing foundation models with efficient SSM-based architectures provides a practical and effective solution for 3D medical image segmentation.

3D分割医学影像Mamba轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。