测试SAM2在视频阴影与镜像检测中的表现，发现其对罕见物体分割效果不佳。
When SAM2 Meets Video Shadow and Mirror Detection
- 用真实标注的点或掩码初始化首帧，后续帧自动预测
- 点提示下性能明显下降，定量与定性均不理想
- 适合研究视频分割泛化能力或罕见目标检测的开发者
作为原始分割一切模型（SAM）的升级版，分割一切模型2（SAM2）不仅提升了图像分割性能，还扩展至视频分割任务。然而，其在视频中罕见物体分割方面的表现尚未充分探索。本研究评估了SAM2在三个不同视频分割任务中的表现，包括视频阴影检测（VSD）和视频镜像检测（VMD）。具体方法为：使用真实标注的点或掩码提示初始化第一帧，随后预测后续帧的掩码。实验结果表明，SAM2在这两项任务上的表现欠佳，尤其在采用点提示时，无论定量还是定性评价均不理想。代码已开源：https://github.com/LeipingJie/SAM2Video。
原文摘要 · Abstract (English)
As the successor to the Segment Anything Model (SAM), the Segment Anything Model 2 (SAM2) not only improves performance in image segmentation but also extends its capabilities to video segmentation. However, its effectiveness in segmenting rare objects that seldom appear in videos remains underexplored. In this study, we evaluate SAM2 on three distinct video segmentation tasks: Video Shadow Detection (VSD) and Video Mirror Detection (VMD). Specifically, we use ground truth point or mask prompts to initialize the first frame and then predict corresponding masks for subsequent frames. Experimental results show that SAM2's performance on these tasks is suboptimal, especially when point prompts are used, both quantitatively and qualitatively. Code is available at \url{https://github.com/LeipingJie/SAM2Video}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。