arXiv:2506.07850cs.CV2025-06被引 1

全自动视频标注系统,无需人工干预即可高效生成精准标注数据。

SAM2Auto: Auto Annotation Using FLASH

  • 结合自动掩码生成与开放世界检测,实现无监督对象识别。
  • 在多帧视频中保持对象一致性,错误率低于现有方法。
  • 适用于多种数据集,无需重训练,适合大规模视频数据构建。

视觉语言模型(VLM)因缺乏标注数据而发展受限,因配对的图文标注需大量人工成本。为突破此瓶颈,我们提出SAM2Auto,首个完全自动化视频数据标注流水线,无需人工介入或特定数据集训练。系统包含两大组件:SMART-OD,一种结合自动掩码生成与开放世界检测能力的鲁棒目标检测系统;FLASH(帧级标注与分割处理器),一种多对象实时视频实例分割(VIS)方法,可在存在间断检测时仍保持对象跨帧一致识别。相比依赖每帧调参且误报率高的现有开放世界检测方法,本系统采用统计策略降低检测误差,确保整段视频中对象追踪的一致性。大量实验表明,SAM2Auto在标注精度上媲美人工标注,同时大幅缩短标注时间并消除人力成本。系统可处理多样化数据集,无需重训练或复杂参数调整,为大规模数据集创建提供实用方案。本工作确立了自动化视频标注新基准,为加速视觉语言理解模型发展提供了突破数据瓶颈的路径。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) lag behind Large Language Models due to the scarcity of annotated datasets, as creating paired visual-textual annotations is labor-intensive and expensive. To address this bottleneck, we introduce SAM2Auto, the first fully automated annotation pipeline for video datasets requiring no human intervention or dataset-specific training. Our approach consists of two key components: SMART-OD, a robust object detection system that combines automatic mask generation with open-world object detection capabilities, and FLASH (Frame-Level Annotation and Segmentation Handler), a multi-object real-time video instance segmentation (VIS) that maintains consistent object identification across video frames even with intermittent detection gaps. Unlike existing open-world detection methods that require frame-specific hyperparameter tuning and suffer from numerous false positives, our system employs statistical approaches to minimize detection errors while ensuring consistent object tracking throughout entire video sequences. Extensive experimental validation demonstrates that SAM2Auto achieves comparable accuracy to manual annotation while dramatically reducing annotation time and eliminating labor costs. The system successfully handles diverse datasets without requiring retraining or extensive parameter adjustments, making it a practical solution for large-scale dataset creation. Our work establishes a new baseline for automated video annotation and provides a pathway for accelerating VLM development by addressing the fundamental dataset bottleneck that has constrained progress in vision-language understanding.

视频标注自动化实例分割VLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。