用检测模型自动生成提示，实现视频分割的全自动化
Det-SAM2:Technical Report on the Self-Prompting Segmentation Framework Based on Segment Anything Model 2
- 用检测模型自动生成对象提示，驱动SAM2完成分割
- 支持无限长视频流处理，内存占用恒定不变
- 适合需要持续自动分割的实时应用，如智能监控
Segment Anything Model 2 (SAM2) 在视频分割和结果优化方面表现出色。我们预计其可进一步发展以实现更高程度的自动化，适用于实际场景。基于SAM2，我们开展了一系列实践，最终构建了一个全自动流水线Det-SAM2：由检测模型自动生成对象提示，供SAM2进行推理与优化。该流水线可在保持原始SAM2效率和精度的前提下，实现对无限长视频流的推理，且显存和内存占用恒定。本技术报告聚焦于Det-SAM2框架的构建及对SAM2的工程优化。我们展示了一个基于该框架的应用案例：基于业务背景的台球赛AI裁判系统。项目地址：https://github.com/motern88/Det-SAM2。
原文摘要 · Abstract (English)
Segment Anything Model 2 (SAM2) demonstrates exceptional performance in video segmentation and refinement of segmentation results. We anticipate that it can further evolve to achieve higher levels of automation for practical applications. Building upon SAM2, we conducted a series of practices that ultimately led to the development of a fully automated pipeline, termed Det-SAM2, in which object prompts are automatically generated by a detection model to facilitate inference and refinement by SAM2. This pipeline enables inference on infinitely long video streams with constant VRAM and RAM usage, all while preserving the same efficiency and accuracy as the original SAM2. This technical report focuses on the construction of the overall Det-SAM2 framework and the subsequent engineering optimization applied to SAM2. We present a case demonstrating an application built on the Det-SAM2 framework: AI refereeing in a billiards scenario, derived from our business context. The project at \url{https://github.com/motern88/Det-SAM2}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。