arXiv:2604.24317cs.CV2026-04被引 3

提出实时视频问答新基准,让模型像助手一样即时响应事件。

Don't Pause! Every prediction matters in a streaming video

论文配图:Don't Pause! Every prediction matters in a streaming video
图 1 · 摘自论文原文
  • 设计多轮主动提问的SPOT-Bench,测试模型实时感知能力。
  • 发现一半视频无响应需求,称为‘死时间’,可节省计算资源。
  • 提出AsynKV,训练零成本适配,兼顾响应速度与准确性。

流式视频模型应即时响应事件,而非事后反馈。现有在线VideoQA基准仍以回溯为主:固定时间点暂停视频,提问关于当前或过去事件,并仅在这些时刻评分。此协议未检验流式预测表现。为此,我们引入SPOT-Bench,包含多轮主动提问,评估通用流式感知与辅助能力,适用于始终在线的实时助手。该基准配备时序精准度指标Timeliness-F1,综合衡量预测的时空精度与全视频覆盖均衡性。实验揭示:(i) 离线模型虽能可靠检测事件,但会无提示频繁输出;(ii) 为减少冗余输出而训练静默模式,却导致响应迟缓;(iii) 半数流式视频无需响应,称作‘死时间’——在此期间计算不影响响应延迟。据此提出AsynKV,一种无需训练的离线模型流式适配方法,保留原事件感知能力,同时优化流式行为。AsynKV采用长短时记忆机制,利用死时间高效调度计算,在SPOT-Bench上优于现有流式模型,并在传统回溯基准上达到顶尖水平。

原文摘要 · Abstract (English)

Streaming video models should respond the moment an event unfolds, not after the moment has passed. Yet existing online VideoQA benchmarks remain largely retrospective. They pause the video at fixed timestamps, pose questions about current or past events, and score models only at those moments. This protocol leaves streaming predictions untested. To close this gap, we introduce SPOT-Bench, featuring multi-turn proactive queries that evaluate general streaming perception and assistive capabilities required by an always-on, real-time assistant. SPOT-Bench comes with Timeliness-F1, a consolidated metric that measures streaming predictions by their temporal precision and balanced coverage across the entire video. Our benchmark reveals: (i) offline models detect events reliably but spam predictions unprompted; (ii) post-training for silence reduces spamming but induces unresponsiveness; (iii) half of the streaming video expects no response, which we term dead-time - compute spent here does not affect response latency. These findings motivate AsynKV, a training-free streaming adaptation of offline models, that retains their event perception while improving their streaming behavior. AsynKV features a long-short term memory, utilized efficiently by scaling compute during dead-time. It serves as a strong baseline on SPOT-Bench, outperforming existing streaming models, and achieves state-of-the-art on retrospective benchmarks.

视频问答流式推理实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。