提出在线视频对象分割新任务，支持实时逐帧推理与语言理解。
Online Reasoning Video Object Segmentation
- 模型仅用当前及历史帧进行逐帧推理，不回看过去结果。
- 在210个视频、1.3万帧上测试，现有方法性能大幅下降。
- 适合需要实时响应的智能监控、人机交互系统使用。
推理视频对象分割需根据自然语言查询生成视频中像素级掩码，可能涉及隐含或时间相关的指代。然而，现有方法均在离线环境下开发与评估，可利用完整视频进行事后消歧，偏离真实应用场景中严格因果、逐帧决策的需求。本文提出在线推理视频对象分割（ORVOS），要求模型仅基于当前和历史帧，增量式理解查询，且无法重新访问过往预测，同时应对事件发展带来的指代变化。为支持评估，我们构建了ORVOSB基准，包含210个视频、12,907个标注帧和512个查询，覆盖五类推理类型，并提供帧级因果标注与指代转移标签。我们进一步提出一个基线模型，采用持续更新的分割提示与结构化时序令牌存储库，在有限计算下实现长时程推理。实验表明，现有方法在严格因果性与指代变化下表现严重退化，而我们的基线为未来研究奠定了坚实基础。
原文摘要 · Abstract (English)
Reasoning video object segmentation predicts pixel-level masks in videos from natural-language queries that may involve implicit and temporally grounded references. However, existing methods are developed and evaluated in an offline regime, where the entire video is available at inference time and future frames can be exploited for retrospective disambiguation, deviating from real-world deployments that require strictly causal, frame-by-frame decisions. We study Online Reasoning Video Object Segmentation (ORVOS), where models must incrementally interpret queries using only past and current frames without revisiting previous predictions, while handling referent shifts as events unfold. To support evaluation, we introduce ORVOSB, a benchmark with frame-level causal annotations and referent-shift labels, comprising 210 videos, 12,907 annotated frames, and 512 queries across five reasoning categories. We further propose a baseline with continually-updated segmentation prompts and a structured temporal token reservoir for long-horizon reasoning under bounded computation. Experiments show that existing methods struggle under strict causality and referent shifts, while our baseline establishes a strong foundation for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。