用状态空间模型替代注意力机制，高效处理脑皮层球面数据
Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation
- 将球面数据转为三角形网格序列，用双向状态空间模型建模
- 推理速度比ViT快4.8倍，内存占用降低91.7%
- 适合资源受限下分析新生儿脑发育的细微模式
基于注意力的方法在建模球面皮层上的长程依赖关系方面表现优异，超越了传统几何深度学习（GDL）模型。然而，其高计算开销和内存需求限制了在大规模数据集上的应用。受计算机视觉中状态空间模型启发，我们提出无需注意力机制的Vision Mamba（Vim），用于球面流形数据分析，构建了一个领域无关的架构。通过将球面数据划分为由细分二十面体生成的三角形补丁序列实现表面分块。所提出的表面Vision Mamba（SiM）在多个新生儿脑皮层表型回归任务上进行了评估。实验表明，SiM优于基于注意力和GDL的方法，在Ico-4网格划分下，推理速度比Surface Vision Transformer（SiT）快4.8倍，内存消耗降低91.7%。敏感性分析进一步证明了其识别细微认知发育模式的潜力。
原文摘要 · Abstract (English)
Attention-based methods have demonstrated exceptional performance in modelling long-range dependencies on spherical cortical surfaces, surpassing traditional Geometric Deep Learning (GDL) models. However, their extensive inference time and high memory demands pose challenges for application to large datasets with limited computing resources. Inspired by the state space model in computer vision, we introduce the attention-free Vision Mamba (Vim) to spherical surfaces, presenting a domain-agnostic architecture for analyzing data on spherical manifolds. Our method achieves surface patching by representing spherical data as a sequence of triangular patches derived from a subdivided icosphere. The proposed Surface Vision Mamba (SiM) is evaluated on multiple neurodevelopmental phenotype regression tasks using cortical surface metrics from neonatal brains. Experimental results demonstrate that SiM outperforms both attention- and GDL-based methods, delivering 4.8 times faster inference and achieving 91.7% lower memory consumption compared to the Surface Vision Transformer (SiT) under the Ico-4 grid partitioning. Sensitivity analysis further underscores the potential of SiM to identify subtle cognitive developmental patterns. The code is available at https://github.com/Rongzhao-He/surface-vision-mamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。