arXiv:2503.19730cs.CVcs.AI2025-03NeurIPS被引 8

提升视频伪装目标分割精度,尤其在简单提示下表现更优。

CamSAM2: Segment Anything Accurately in Camouflaged Videos

  • 引入去伪装令牌与融合模块,增强对伪装场景的特征适应能力。
  • 在MoCA-Mask上点击提示提升12.2点mDice,遮罩提示提升19.6点mDice。
  • 无需修改SAM2参数,新增参数极少,适合快速集成到现有系统中。

视频伪装目标分割(VCOS)旨在识别与环境无缝融合的伪装物体,具有广泛的实际应用价值。随着SAM2的发布,视频分割取得显著进展,但其在处理伪装视频时表现不佳,尤其在点、框等简单提示下。为此,我们提出CamSAM2,通过不修改SAM2参数的方式增强其处理伪装场景的能力。具体地,引入去伪装令牌以灵活调整特征;设计隐式与显式对象感知融合模块,充分利用当前帧和前序帧的细粒度高分辨率特征;提出对象原型生成(OPG)模块,利用前序帧高质量特征抽象并记忆对象原型。大量实验验证了方法有效性:尽管仅增加极少可学习参数,CamSAM2在三个VCOS数据集上显著优于SAM2,尤其在使用Hiera-T作为骨干网络时,在MoCA-Mask上点击提示实现12.2 mDice提升,在SUN-SEG-Hard上遮罩提示实现19.6 mDice提升。代码已开源。

原文摘要 · Abstract (English)

Video camouflaged object segmentation (VCOS), aiming at segmenting camouflaged objects that seamlessly blend into their environment, is a fundamental vision task with various real-world applications. With the release of SAM2, video segmentation has witnessed significant progress. However, SAM2's capability of segmenting camouflaged videos is suboptimal, especially when given simple prompts such as point and box. To address the problem, we propose Camouflaged SAM2 (CamSAM2), which enhances SAM2's ability to handle camouflaged scenes without modifying SAM2's parameters. Specifically, we introduce a decamouflaged token to provide the flexibility of feature adjustment for VCOS. To make full use of fine-grained and high-resolution features from the current frame and previous frames, we propose implicit object-aware fusion (IOF) and explicit object-aware fusion (EOF) modules, respectively. Object prototype generation (OPG) is introduced to abstract and memorize object prototypes with informative details using high-quality features from previous frames. Extensive experiments are conducted to validate the effectiveness of our approach. While CamSAM2 only adds negligible learnable parameters to SAM2, it substantially outperforms SAM2 on three VCOS datasets, especially achieving 12.2 mDice gains with click prompt on MoCA-Mask and 19.6 mDice gains with mask prompt on SUN-SEG-Hard, with Hiera-T as the backbone. The code is available at https://github.com/zhoustan/CamSAM2.

视频分割伪装检测SAM2

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。