用免费视频帧和粗略标注,降低语义分割的标注成本。
Can Unsupervised Segmentation Reduce Annotation Costs for Video Semantic Segmentation?
- 用SAM系列模型自动生成分割掩码,减少人工标注。
- 仅需原标注量三分之二,性能基本不变。
- 数据多样性比数量更重要,适合资源有限的研究者。
当前视频语义分割的深度神经网络需要大量精细的像素级标注才能达到最佳效果,但获取这些标注成本极高。相比之下,原始未标注视频帧几乎免费获取,而粗略标注(无需精确边界)也便宜得多。本文研究如何利用这些低成本资源降低视频分割数据集的标注成本。结果表明,通过使用先进的分割基础模型——Segment Anything Model (SAM) 和 Segment Anything Model 2 (SAM 2),结合未标注帧与粗略标注,可自动化生成掩码,显著减少人工标注工作量。实验显示,若合理使用,标注需求可减少三分之一,同时保持相近的分割性能。更重要的是,分析发现数据集中的帧多样性比帧数多少对性能影响更大。
原文摘要 · Abstract (English)
Present-day deep neural networks for video semantic segmentation require a large number of fine-grained pixel-level annotations to achieve the best possible results. Obtaining such annotations, however, is very expensive. On the other hand, raw, unannotated video frames are practically free to obtain. Similarly, coarse annotations, which do not require precise boundaries, are also much cheaper. This paper investigates approaches to reduce the annotation cost required for video segmentation datasets by utilising such resources. We show that using state-of-the-art segmentation foundation models, Segment Anything Model (SAM) and Segment Anything Model 2 (SAM 2), we can utilise both unannotated frames as well as coarse annotations to alleviate the effort required for manual annotation of video segmentation datasets by automating mask generation. Our investigation suggests that if used appropriately, we can reduce the need for annotation by a third with similar performance for video semantic segmentation. More significantly, our analysis suggests that the variety of frames in the dataset is more important than the number of frames for obtaining the best performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。