MambaBack提升病理切片分析,兼顾局部细节与全局上下文。
MambaBack: Bridging Local Features and Global Contexts in Whole Slide Image Analysis

- 用希尔伯特采样保留切片二维空间结构,增强模型空间感知。
- 融合门控CNN与双向Mamba块,实现多尺度特征提取与上下文建模。
- 异步分块设计降低推理内存峰值,适合边缘设备部署。
全切片图像(WSI)分析在计算病理学中至关重要,通过整合不同放大倍数下的形态与结构信息实现癌症诊断。多实例学习(MIL)是其标准框架。近期,源自自然语言处理的Mamba因其高效性与全局上下文建模能力,超越Transformer成为MIL骨干。然而现有Mamba-based方法面临三大挑战:(1) 1D序列展平破坏2D空间局部性;(2) 细粒度细胞结构建模不足;(3) 在资源受限边缘设备上推理时内存峰值过高。研究表明,Mamba的SSM模块在局部特征提取中冗余,门控CNN已足够。鉴于WSI分析既需类自然图像的细粒度局部特征,又需类NLP的全局上下文,我们提出MambaBack——一种融合Mamba与MambaOut优势的新型混合架构。首先,采用希尔伯特采样策略,在1D序列中保留瓦片的2D空间局部性,提升空间感知。其次,设计分层结构:基于MambaOut的1D门控CNN块捕捉局部细胞特征,双向BiMamba2块聚合全局上下文,联合增强多尺度表示。最后,引入非对称分块设计,训练时并行处理,推理时分块流式累积,显著降低峰值内存。五个数据集上的实验表明,MambaBack超越七种前沿方法。源代码与数据集公开可用。
原文摘要 · Abstract (English)
Whole Slide Image (WSI) analysis is pivotal in computational pathology, enabling cancer diagnosis by integrating morphological and architectural cues across magnifications. Multiple Instance Learning (MIL) serves as the standard framework for WSI analysis. Recently, Mamba has become a promising backbone for MIL, overtaking Transformers due to its efficiency and global context modeling capabilities originating from Natural Language Processing (NLP). However, existing Mamba-based MIL approaches face three critical challenges: (1) disruption of 2D spatial locality during 1D sequence flattening; (2) sub-optimal modeling of fine-grained local cellular structures; and (3) high memory peaks during inference on resource-constrained edge devices. Studies like MambaOut reveal that Mamba's SSM component is redundant for local feature extraction, where Gated CNNs suffice. Recognizing that WSI analysis demands both fine-grained local feature extraction akin to natural images, and global context modeling akin to NLP, we propose MambaBack, a novel hybrid architecture that harmonizes the strengths of Mamba and MambaOut. First, we propose the Hilbert sampling strategy to preserve the 2D spatial locality of tiles within 1D sequences, enhancing the model's spatial perception. Second, we design a hierarchical structure comprising a 1D Gated CNN block based on MambaOut to capture local cellular features, and a BiMamba2 block to aggregate global context, jointly enhancing multi-scale representation. Finally, we implement an asymmetric chunking design, allowing parallel processing during training and chunking-streaming accumulation during inference, minimizing peak memory usage for deployment. Experimental results on five datasets demonstrate that MambaBack outperforms seven state-of-the-art methods. Source code and datasets are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。