提出跨模态脑肿瘤分割新框架,提升精度与实用性。
Synergistic Modality-and-Slice Memory Framework for Cross-Modal 3D Brain Tumor Segmentation
- 双记忆机制融合多模态与切片间信息
- 在多个数据集上达到最优分割效果
- 无需特定类别提示,适合临床实用
三维多模态脑肿瘤分割对多模态医疗至关重要,需精准识别内部解剖亚区。现有基于提示的分割方法虽支持交互,但忽略跨模态关联,依赖耗时的类别特定提示,限制实际应用。为此,我们提出MSM-Seg框架,采用协同双记忆分割范式,结合多模态与切片间信息,并引入高效无类别提示机制。首先设计模态-切片记忆注意力(MSMA)以挖掘输入扫描间的复杂跨模态相关性与空间切片依赖性;其次提出多尺度无类别提示编码器(MCP-Encoder),为解码提供全肿瘤区域引导;此外设计模态自适应融合解码器(MF-Decoder),利用不同模态间的互补解码信息提升分割精度。在多个MRI数据集上的实验表明,该框架在多模态转移瘤与胶质瘤分割任务中均优于现有最佳方法。代码已开源:https://github.com/xq141839/MSM-Seg。
原文摘要 · Abstract (English)
The 3D multi-modal brain tumor segmentation is critical to multi-modal healthcare, and it requires accurate identification of distinct internal anatomical subregions. While the recent prompt-based segmentation paradigms enable interactive experiences for clinicians, existing methods ignore cross-modal correlations and rely on labor-intensive category-specific prompts, limiting their applicability in real-world scenarios. To address these issues, we propose the MSM-Seg, a synergistic framework for multi-modal brain tumor segmentation. The MSM-Seg introduces a dual-memory segmentation paradigm that synergistically integrates multi-modal and inter-slice information with an efficient category-agnostic prompt for brain tumor understanding. To this end, we first devise a modality-and-slice memory attention (MSMA) to exploit the complex cross-modal correlations and spatial-slice dependencies among the input scans. \cz{Then, we propose a multi-scale category-agnostic prompt encoder (MCP-Encoder) to provide whole tumor region guidance for decoding.} Moreover, we devise a modality-adaptive fusion decoder (MF-Decoder) that leverages the complementary decoding information across different modalities to improve segmentation accuracy. Extensive experiments on different MRI datasets demonstrate that our MSM-Seg framework outperforms state-of-the-art methods in multi-modal metastases and glioma tumor segmentation. The code is available at https://github.com/xq141839/MSM-Seg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。