提出多模态感知融合网络，提升遥感图像文本定位分割精度
Multimodal-Aware Fusion Network for Referring Remote Sensing Image Segmentation
- 通过相关性融合模块增强多尺度视觉特征
- 在RRSIS-D数据集上优于现有方法，显著提升分割准确率
- 适合遥感目标识别与多模态理解研究者参考
指代式遥感图像分割（RRSIS）是一项新型遥感图像分割任务，旨在根据给定文本描述分割对应目标，具有重要应用价值。以往方法通过显式特征交互融合视觉与语言模态，难以充分挖掘双分支编码器中的多模态信息。本文设计了多模态感知融合网络（MAFN），实现两模态间的细粒度对齐与融合。提出相关性融合模块（CFM），通过引入自适应噪声增强Transformer中的多尺度视觉特征，并整合跨模态感知特征；同时采用多尺度精炼卷积（MSRC），适应不同尺度下目标的多种方向，提升表征能力。大量实验表明，MAFN在RRSIS-D数据集上显著优于当前最优方法。源代码已开源：https://github.com/Roaxy/MAFN。
原文摘要 · Abstract (English)
Referring remote sensing image segmentation (RRSIS) is a novel visual task in remote sensing images segmentation, which aims to segment objects based on a given text description, with great significance in practical application. Previous studies fuse visual and linguistic modalities by explicit feature interaction, which fail to effectively excavate useful multimodal information from dual-branch encoder. In this letter, we design a multimodal-aware fusion network (MAFN) to achieve fine-grained alignment and fusion between the two modalities. We propose a correlation fusion module (CFM) to enhance multi-scale visual features by introducing adaptively noise in transformer, and integrate cross-modal aware features. In addition, MAFN employs multi-scale refinement convolution (MSRC) to adapt to the various orientations of objects at different scales to boost their representation ability to enhances segmentation accuracy. Extensive experiments have shown that MAFN is significantly more effective than the state of the art on RRSIS-D datasets. The source code is available at https://github.com/Roaxy/MAFN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。