用光流与开放词汇检测实现零样本视频伪装目标分割,性能大幅提升。
ZS-VCOS: Zero-Shot Video Camouflaged Object Segmentation By Optical Flow and Open Vocabulary Object Detection
- 结合光流与开放词汇检测,构建无需训练的模块化分割流程。
- 在MoCA-Mask上将F-measure提升至0.628,超越现有零样本方法。
- 适用于医疗、工业检测等需快速识别隐蔽目标的场景。
伪装目标分割相比传统分割任务更具挑战性,主要源于伪装物体与背景在图案和颜色上高度相似。有效解决方案在害虫控制、缺陷检测及医学影像病变分割等领域具有重要意义。以往研究多聚焦于有监督或无监督预训练方法,零样本方法发展不足。现有零样本技术常依赖SAM自动模式或视觉语言模型生成提示,但受限于物体与背景相似性,性能不佳。本文提出一种不依赖训练的方法,通过整合SAM-2与Owl-v2,并引入时序信息构建模块化管道。在MoCA-Mask数据集上,该方法将加权F-measure($F_β^w$)从0.296提升至0.628,显著优于现有零样本方法;同时超越部分有监督方法,使F-measure由0.476提升至0.628。在MoCA-Filter数据集上,成功率达0.697,高于FlowSAM(0.628)。消融实验验证各组件贡献。此外,本文指出前人研究在指标与设置上的不一致问题。代码已开源:https://github.com/weathon/vcos。
原文摘要 · Abstract (English)
Camouflaged object segmentation presents unique challenges compared to traditional segmentation tasks, primarily due to the high similarity in patterns and colors between camouflaged objects and their backgrounds. Effective solutions to this problem have significant implications in critical areas such as pest control, defect detection, and lesion segmentation in medical imaging. Prior research has predominantly emphasized supervised or unsupervised pre-training methods, leaving zero-shot approaches significantly underdeveloped. Existing zero-shot techniques commonly utilize the Segment Anything Model (SAM) in automatic mode or rely on vision-language models to generate cues for segmentation; however, their performances remain unsatisfactory, due to the similarity of the camouflaged object and the background. This work studies how to avoid training by integrating large pre-trained models like SAM-2 and Owl-v2 with temporal information into a modular pipeline. Evaluated on the MoCA-Mask dataset, our approach achieves outstanding performance improvements, significantly outperforming existing zero-shot methods by raising the F-measure ($F_β^w$) from 0.296 to 0.628. Our approach also surpasses supervised methods, increasing the F-measure from 0.476 to 0.628. Additionally, evaluation on the MoCA-Filter dataset demonstrates an increase in the success rate from 0.628 to 0.697 when compared with FlowSAM, a supervised transfer method. A thorough ablation study further validates the individual contributions of each component. Besides our main contributions, we also highlight inconsistencies in previous work regarding metrics and settings. Code can be found in https://github.com/weathon/vcos.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。