用几何提示提升遥感图像开放词汇分割精度
RSGPNet: Geometric Prompting for Remote Sensing Open-Vocabulary Semantic Segmentation

- 通过文本引导粗分割+几何重提示,迭代优化分割结果
- 在多个遥感数据集上显著优于现有方法,边界区域更准确
- 无需训练,适合需要可解释性的遥感分析场景
开放词汇语义分割(OVSS)实现了文本引导的未见物体分割,突破了固定类别限制,实现开放世界理解。然而,现有方法主要聚焦于修改CLIP注意力机制,在遥感(RS)领域仍存在局部分割不稳定的问题。为此,本文提出RSGPNet,一种无需训练的遥感开放词汇分割几何提示框架,通过利用物体几何区域和一致性约束来优化分割结果。具体包含三个核心模块:文本引导粗分割模块(TCM)、几何重提示模块(GRP)和粗到细一致性验证机制(CVM)。TCM结合文本提示与输入图像生成初始粗分割掩码;GRP将这些粗掩码转换为几何框提示并反馈至分割模型以生成精炼掩码;最后,CVM通过一致性计算防止错误区域被强化。该框架有效提升了复杂区域(如类别边界)的分割精度。大量实验表明,RSGPNet在多个遥感数据集上均显著优于当前最优方法,且具有优异可解释性。代码已开源。
原文摘要 · Abstract (English)
Open-vocabulary semantic segmentation (OVSS) enables text-guided segmentation of unseen objects, breaking fixed-class limitations to achieve open-world understanding. However, existing OVSS methods primarily focus on modifying the CLIP attention mechanism, which still suffers from unstable local segmentation for remote sensing (RS) domain. To address these limitations, we propose RSGPNet, a training-free geometric prompting framework for RS OVSS that refines segmentation by leveraging object geometric areas and consistency constraints. Specifically, RSGPNet comprises three core modules: a Text-guided Coarse Mask module (TCM), a Geometric Re-prompting Module (GRP), and a Coarse-to-fine Consistency Verification Mechanism (CVM). TCM utilizes text prompts and the input image to construct initial coarse segmentation masks. GRP then converts these coarse masks into geometric box prompts, feeding them back into the segmentation model to generate refined masks. Finally, CVM employs consistency computation to prevent prompting from reinforcing erroneous regions. They allow the model to improve segmentation accuracy in complex areas, such as category boundaries. Extensive experiments on RS datasets demonstrate that RSGPNet significantly outperforms state-of-the-art methods across both quantitative and qualitative metrics while exhibiting excellent interpretability. The code is released at \href{https://github.com/wangshanwen001/RSGPNet}{https://github.com/wangshanwen001/RSGPNet}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。