提出STRIDE模型，让视频流能自动判断何时该回应，更准更连贯。
STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding
- 将响应时机建模为带滑动窗口的序列结构问题
- 用轻量扩散模块迭代优化激活信号，提升时序一致性
- 适合需要实时交互的视频流应用，如直播分析
视频大语言模型在长视频离线推理上表现强劲，但实际应用中更需在线感知与主动交互——视频帧持续到来，系统不仅要决定如何回答，还需判断何时回应。本文将主动触发机制重新视为结构化序列建模问题，观察到视频流中的时间过渡自然形成分段式激活模式。为此，提出STRIDE（结构化时序精炼与迭代去噪），在激活接口引入轻量级掩码扩散模块，对滑动时间窗内的激活信号进行联合预测与渐进式修正。在多个流式基准和下游模型上的实验表明，STRIDE能生成更可靠、时序更连贯的主动响应，在在线流场景中显著提升‘何时回应’的决策质量。
原文摘要 · Abstract (English)
Recent progress in video large language models (Video-LLMs) has enabled strong offline reasoning over long and complex videos. However, real-world deployments increasingly require streaming perception and proactive interaction, where video frames arrive online and the system must decide not only what to respond, but also when to respond. In this work, we revisit proactive activation in streaming video as a structured sequence modeling problem, motivated by the observation that temporal transitions in streaming video naturally form span-structured activation patterns. To capture this span-level structure, we model activation signals jointly over a sliding temporal window and update them iteratively as new frames arrive. We propose STRIDE (Structured Temporal Refinement with Iterative DEnoising), which employs a lightweight masked diffusion module at the activation interface to jointly predict and progressively refine activation signals across the window. Extensive experiments on diverse streaming benchmarks and downstream models demonstrate that STRIDE shows more reliable and temporally coherent proactive responses, significantly improving when-to-speak decision quality in online streaming scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。