统一有提示与无提示视频分割，实现端到端自动分割
Autoregressive Universal Video Segmentation Model
- 将视频分割类比语言建模，用自回归方式逐帧预测掩码
- 在6个基准上超越现有方法，16帧序列训练速度提升2.5倍
- 适合需要全自动对象追踪的实时视频应用
近期视频基础模型如SAM2通过将掩码视为通用范式，在提示引导的视频分割中表现优异。然而，许多真实场景需要无提示分割——即在无外部提示下检测并追踪视频中所有物体，导致当前方案分散于任务专用模型与流程中。本文将流式视频分割重构为序列掩码预测问题，类比语言建模，提出自回归通用分割模型（AUSM），单一架构同时支持提示与无提示分割。基于最新状态空间模型，AUSM维护固定大小的空间状态，可扩展至任意长度视频流。此外，所有组件均设计为跨帧并行训练，相较迭代训练大幅提速。在标准基准（DAVIS17、YouTube-VOS 2018 & 2019、MOSE、YouTube-VIS 2019 & 2021、OVIS）上，AUSM优于以往通用流式视频分割方法，并在16帧序列上实现最高2.5倍训练加速。
原文摘要 · Abstract (English)
Recent video foundation models such as SAM2 excel at prompted video segmentation by treating masks as a general-purpose primitive. However, many real-world settings require unprompted segmentation that aims to detect and track all objects in a video without external cues, leaving today's landscape fragmented across task-specific models and pipelines. We recast streaming video segmentation as sequential mask prediction, analogous to language modeling, and introduce the Autoregressive Universal Segmentation Model (AUSM), a single architecture that unifies both prompted and unprompted video segmentation. Built on recent state-space models, AUSM maintains a fixed-size spatial state and scales to video streams of arbitrary length. Furthermore, all components of AUSM are designed for parallel training across frames, yielding substantial speedups over iterative training. On standard benchmarks (DAVIS17, YouTube-VOS 2018 & 2019, MOSE, YouTube-VIS 2019 & 2021, and OVIS) AUSM outperforms prior universal streaming video segmentation methods and achieves up to 2.5x faster training on 16-frame sequences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。