arXiv:2509.00677cs.CV2025-09被引 1

用Mamba提升遥感多模态分类效率,融合更准更快。

CSFMamba: Cross State Fusion Mamba Operator for Multimodal Remote Sensing Image Classification

  • 设计跨状态Mamba模块,直接融合高光谱与激光雷达特征。
  • 在MUUFL和Houston2018上准确率超Transformer,训练负担更低。
  • 适合需要高效多模态遥感分类的科研与工程应用。

多模态融合在遥感图像分类中取得显著进展,能有效利用空间-光谱信息的互补性。尽管深度学习方法如CNN和Transformer广泛应用,但其面临二次计算复杂度问题,难以建模长距离依赖。状态空间模型(SSM)通过引入时变参数并支持硬件优化解决了此问题。然而,现有Mamba架构无法直接进行特征融合。为此,本文提出跨状态融合Mamba(CSFMamba)网络:首先针对Mamba结构设计遥感数据预处理模块,并结合CNN提取多层次特征;其次创新设计基于Mamba算子的跨状态融合模块,实现双模态特征深度融合;通过融合Mamba低计算开销与CNN强表达能力,构建更强大的主干网络,显著提升对高光谱(HSI)与激光雷达(LiDAR)模态间融合关系的全局理解。在MUUFL与Houston2018两个数据集上的实验表明,该方法在降低网络训练负担的前提下,性能优于基于Transformer的基准模型。

原文摘要 · Abstract (English)

Multimodal fusion has made great progress in the field of remote sensing image classification due to its ability to exploit the complementary spatial-spectral information. Deep learning methods such as CNN and Transformer have been widely used in these domains. State Space Models recently highlighted that prior methods suffer from quadratic computational complexity. As a result, modeling longer-range dependencies of spatial-spectral features imposes an overwhelming burden on the network. Mamba solves this problem by incorporating time-varying parameters into ordinary SSM and performing hardware optimization, but it cannot perform feature fusion directly. In order to make full use of Mamba's low computational burden and explore the potential of internal structure in multimodal feature fusion, we propose Cross State Fusion Mamba (CSFMamba) Network. Specifically, we first design the preprocessing module of remote sensing image information for the needs of Mamba structure, and combine it with CNN to extract multi-layer features. Secondly, a cross-state module based on Mamba operator is creatively designed to fully fuse the feature of the two modalities. The advantages of Mamba and CNN are combined by designing a more powerful backbone. We capture the fusion relationship between HSI and LiDAR modalities with stronger full-image understanding. The experimental results on two datasets of MUUFL and Houston2018 show that the proposed method outperforms the experimental results of Transformer under the premise of reducing the network training burden.

遥感分类多模态融合Mamba

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。