arXiv:2505.04905cs.CV2025-05ECCV被引 6

用网格点提示SAM,实现弱监督目标定位的高精度分割。

Pro2SAM: Mask Prompt to SAM with Grid Points for Weakly Supervised Object Localization

  • 用全局令牌变压器生成粗粒度前景图作掩码提示
  • 以网格点密集提示SAM,提升前景区域激活概率
  • 像素级相似度匹配,定位准确率达84.03%(CUB)

弱监督目标定位(WSOL)仅需图像级标签即可定位物体,因标注成本低备受关注。现有方法依赖CNN的类激活图(CAM)或Transformer的自注意力图,但难以捕捉前景物体的像素级细粒度信息,限制了性能提升。为此,本文首次利用分割一切模型(SAM)的零样本泛化与细粒度分割能力,增强完整物体区域的激活。针对单点提示导致语义模糊的问题,提出基于网格点的掩码提示网络Pro2SAM。首先设计全局令牌变换器(GTFormer),联合嵌入块令牌与新型全局令牌,生成粗粒度前景图作为灵活掩码提示;其次将网格点作为密集提示输入SAM,最大化前景掩码概率,避免单点/框提示造成的漏检;最后提出像素级相似度度量,实现从掩码提示到SAM的匹配,得分最高的掩码即为最终定位图。实验表明,Pro2SAM在CUB-200-2011和ILSVRC上分别达到84.03%和66.85%的Top-1定位准确率,达到当前最优水平。

原文摘要 · Abstract (English)

Weakly Supervised Object Localization (WSOL), which aims to localize objects by only using image-level labels, has attracted much attention because of its low annotation cost in real applications. Current studies focus on the Class Activation Map (CAM) of CNN and the self-attention map of transformer to identify the region of objects. However, both CAM and self-attention maps can not learn pixel-level fine-grained information on the foreground objects, which hinders the further advance of WSOL. To address this problem, we initiatively leverage the capability of zero-shot generalization and fine-grained segmentation in Segment Anything Model (SAM) to boost the activation of integral object regions. Further, to alleviate the semantic ambiguity issue accrued in single point prompt-based SAM, we propose an innovative mask prompt to SAM (Pro2SAM) network with grid points for WSOL task. First, we devise a Global Token Transformer (GTFormer) to generate a coarse-grained foreground map as a flexible mask prompt, where the GTFormer jointly embeds patch tokens and novel global tokens to learn foreground semantics. Secondly, we deliver grid points as dense prompts into SAM to maximize the probability of foreground mask, which avoids the lack of objects caused by a single point/box prompt. Finally, we propose a pixel-level similarity metric to come true the mask matching from mask prompt to SAM, where the mask with the highest score is viewed as the final localization map. Experiments show that the proposed Pro2SAM achieves state-of-the-art performance on both CUB-200-2011 and ILSVRC, with 84.03\% and 66.85\% Top-1 Loc, respectively.

弱监督目标定位SAM网格提示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。