arXiv:2508.03008eess.IVcs.AI2025-08中稿 · MICCAI MLMI 2025 W…被引 1

用Mamba模型融合多模态脑影像,实现高效实时诊断。

ClinicalFMamba: Advancing Clinical Assessment using Mamba-based Multimodal Neuroimaging Fusion

  • 结合CNN局部特征与Mamba全局建模能力,支持2D/3D图像融合。
  • 在三个数据集上超越基线方法,实现端到端实时融合。
  • 适合临床部署,提升脑肿瘤分类准确率,适用于医疗影像分析。

多模态医学图像融合通过整合不同成像模态的互补信息,提升诊断精度与治疗规划效果。尽管深度学习已推动性能进步,现有方法仍存在局限:卷积神经网络(CNN)擅长局部特征提取,但难以建模全局上下文;而变换器(Transformers)虽能有效捕捉长程依赖,却因二次计算复杂度限制了临床应用。最近的状态空间模型(SSMs)提供新路径,可通过选择性扫描机制以线性时间实现高效长程依赖建模。然而,其在三维体数据上的扩展及临床验证仍不充分。本文提出ClinicalFMamba,一种端到端的CNN-Mamba混合架构,协同实现2D与3D图像的局部与全局特征建模,并设计三平面扫描策略以有效学习三维体数据的依赖关系。在三个数据集上的综合评估显示,该方法在多个定量指标上表现更优,且实现近实时融合。进一步在下游2D/3D脑肿瘤分类任务中验证了其临床价值,性能显著优于基线方法。本方法建立了一种适用于实时临床部署的高效多模态医学图像融合新范式。

原文摘要 · Abstract (English)

Multimodal medical image fusion integrates complementary information from different imaging modalities to enhance diagnostic accuracy and treatment planning. While deep learning methods have advanced performance, existing approaches face critical limitations: Convolutional Neural Networks (CNNs) excel at local feature extraction but struggle to model global context effectively, while Transformers achieve superior long-range modeling at the cost of quadratic computational complexity, limiting clinical deployment. Recent State Space Models (SSMs) offer a promising alternative, enabling efficient long-range dependency modeling in linear time through selective scan mechanisms. Despite these advances, the extension to 3D volumetric data and the clinical validation of fused images remains underexplored. In this work, we propose ClinicalFMamba, a novel end-to-end CNN-Mamba hybrid architecture that synergistically combines local and global feature modeling for 2D and 3D images. We further design a tri-plane scanning strategy for effectively learning volumetric dependencies in 3D images. Comprehensive evaluations on three datasets demonstrate the superior fusion performance across multiple quantitative metrics while achieving real-time fusion. We further validate the clinical utility of our approach on downstream 2D/3D brain tumor classification tasks, achieving superior performance over baseline methods. Our method establishes a new paradigm for efficient multimodal medical image fusion suitable for real-time clinical deployment.

多模态融合医学影像Mamba脑肿瘤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。