用语义定位引导SAM,实现遥感图像精准分割
Semantic Localization Guiding Segment Anything Model For Reference Remote Sensing Image Segmentation
- 分两阶段:先粗定位再精分割,降低复杂场景干扰
- 无需训练即可实现高精度分割,减少标注数据需求
- 适合需要少标注的遥感图像分割任务,尤其关注特定目标
参考遥感图像分割(RRSIS)任务根据文本描述生成图像中指定对象的分割掩码,受到广泛关注。现有方法依赖多模态融合主干和语义分割头,面临密集标注与复杂场景理解挑战。为此,我们提出一种名为提示生成语义定位引导的SAM(PSLG-SAM)框架,将任务分解为粗定位与细分割两阶段:第一阶段通过视觉定位网络粗略定位文本描述对象;第二阶段利用第一阶段坐标引导增强版SAM,结合基于聚类的前景点生成器和边界迭代优化策略实现精确分割。值得注意的是,第二阶段可免训练,显著减轻标注负担。此外,分阶段设计使模型专注特定区域,避免复杂场景干扰。我们还构建了一个高质量、多类别人工标注数据集。在两个数据集(RRSIS-D 和 RRSIS-M)上的实验表明,PSLG-SAM性能显著提升,优于现有最先进模型。代码将公开。
原文摘要 · Abstract (English)
The Reference Remote Sensing Image Segmentation (RRSIS) task generates segmentation masks for specified objects in images based on textual descriptions, which has attracted widespread attention and research interest. Current RRSIS methods rely on multi-modal fusion backbones and semantic segmentation heads but face challenges like dense annotation requirements and complex scene interpretation. To address these issues, we propose a framework named \textit{prompt-generated semantic localization guiding Segment Anything Model}(PSLG-SAM), which decomposes the RRSIS task into two stages: coarse localization and fine segmentation. In coarse localization stage, a visual grounding network roughly locates the text-described object. In fine segmentation stage, the coordinates from the first stage guide the Segment Anything Model (SAM), enhanced by a clustering-based foreground point generator and a mask boundary iterative optimization strategy for precise segmentation. Notably, the second stage can be train-free, significantly reducing the annotation data burden for the RRSIS task. Additionally, decomposing the RRSIS task into two stages allows for focusing on specific region segmentation, avoiding interference from complex scenes.We further contribute a high-quality, multi-category manually annotated dataset. Experimental validation on two datasets (RRSIS-D and RRSIS-M) demonstrates that PSLG-SAM achieves significant performance improvements and surpasses existing state-of-the-art models.Our code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。