arXiv:2501.13426cs.CV2025-01被引 18

用自动提示技术让SAM模型精准提取滑坡,仅需图像标签就行

Auto-Prompting SAM for Weakly Supervised Landslide Extraction

  • 基于类激活图自动生成混合提示,直接指导SAM分割
  • 在高分辨率遥感数据上,F1提升3.0%,IoU提升3.69%
  • 适合缺乏像素级标注的滑坡监测任务,代码开源

弱监督滑坡提取旨在仅使用图像级别标签训练模型,从遥感影像中识别滑坡区域。然而,由于缺乏像素级监督及滑坡自身特性,常导致分割边界不精确。为此,本文提出一种简单有效的自动提示方法APSAM,通过提示工程直接从SAM推理中生成细粒度分割掩码。该方法利用目标定位网络生成的类激活图(CAMs),自适应地构建包含框提示与点提示的混合提示。设计的自适应提示生成(APG)算法充分挖掘CAM视觉模式,高效生成用于滑坡提取的伪掩码。实验在高分辨率航空与卫星数据集上验证了方法有效性,相比现有先进方法,F1分数提升至少3.0%,交并比(IoU)提升3.69%。代码与数据集将公开于https://github.com/zxk688。

原文摘要 · Abstract (English)

Weakly supervised landslide extraction aims to identify landslide regions from remote sensing data using models trained with weak labels, particularly image-level labels. However, it is often challenged by the imprecise boundaries of the extracted objects due to the lack of pixel-wise supervision and the properties of landslide objects. To tackle these issues, we propose a simple yet effective method by auto-prompting the Segment Anything Model (SAM), i.e., APSAM. Instead of depending on high-quality class activation maps (CAMs) for pseudo-labeling or fine-tuning SAM, our method directly yields fine-grained segmentation masks from SAM inference through prompt engineering. Specifically, it adaptively generates hybrid prompts from the CAMs obtained by an object localization network. To provide sufficient information for SAM prompting, an adaptive prompt generation (APG) algorithm is designed to fully leverage the visual patterns of CAMs, enabling the efficient generation of pseudo-masks for landslide extraction. These informative prompts are able to identify the extent of landslide areas (box prompts) and denote the centers of landslide objects (point prompts), guiding SAM in landslide segmentation. Experimental results on high-resolution aerial and satellite datasets demonstrate the effectiveness of our method, achieving improvements of at least 3.0\% in F1 score and 3.69\% in IoU compared to other state-of-the-art methods. The source codes and datasets will be available at https://github.com/zxk688.

滑坡提取弱监督SAM提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。