提出新模型融合多光谱与全色遥感图像,提升分类精度。
A Diff-Attention Aware State Space Fusion Model for Remote Sensing Classification
- 设计跨模态差异注意力模块分离共性与特有特征
- 通过保持空间结构的视觉状态空间模型保留局部细节
- 用注意力加权线性融合处理语义差异大的特征
多光谱(MS)和全色(PAN)图像描述同一地表,各自具备优势并包含大量相似信息。为分离共性信息与各自优势,减少融合阶段的特征冗余,本文提出一种差分注意力感知的状态空间融合模型(DAS2F-Model)。基于选择性状态空间模型,设计了跨模态差分注意力模块(CMDA-Module),用于提取并分离MS与PAN图像的共有特征及其主导特征。其中,空间保持视觉马尔可夫(SPVM)通过合理优化视觉马尔可夫输入,保留图像空间结构并捕捉局部特征。考虑到融合阶段特征分离后存在显著语义差异,简单融合方法难以有效整合,因此提出注意力感知线性融合模块(AALF-Module),通过计算影响系数实现像素级线性融合,可在保持特征尺寸不变的同时融合语义差异较大的特征。实证评估表明,该方法优于现有方法。代码已开源:https://github.com/AVKSKVL/DAS-F-Model
原文摘要 · Abstract (English)
Multispectral (MS) and panchromatic (PAN) images describe the same land surface, so these images not only have their own advantages, but also have a lot of similar information. In order to separate these similar information and their respective advantages, reduce the feature redundancy in the fusion stage. This paper introduces a diff-attention aware state space fusion model (DAS2F-Model) for multimodal remote sensing image classification. Based on the selective state space model, a cross-modal diff-attention module (CMDA-Module) is designed to extract and separate the common features and their respective dominant features of MS and PAN images. Among this, space preserving visual mamba (SPVM) retains image spatial features and captures local features by optimizing visual mamba's input reasonably. Considering that features in the fusion stage will have large semantic differences after feature separation and simple fusion operations struggle to effectively integrate these significantly different features, an attention-aware linear fusion module (AALF-Module) is proposed. It performs pixel-wise linear fusion by calculating influence coefficients. This mechanism can fuse features with large semantic differences while keeping the feature size unchanged. Empirical evaluations indicate that the presented method achieves better results than alternative approaches. The relevant code can be found at:https://github.com/AVKSKVL/DAS-F-Model
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。