仅用最近4帧视频，简单模型就超越复杂流式视频理解方法。
A Simple Baseline for Streaming Video Understanding
- 用滑动窗口只传最近4帧给现成视觉语言模型，即达高性能。
- 在OVO-Bench上准确率67.7%，StreamingBench上80.59%。
- 揭示感知与记忆的权衡，适合评估流式视频模型新基准。
当前流式视频理解方法越来越依赖复杂的记忆机制来处理长视频流。我们挑战这一趋势：一个仅将最近N帧输入现成视觉语言模型的简单滑动窗口基线，已能匹配或超越已有流式模型。我们将其形式化为SimpleStream，并在OVO-Bench和StreamingBench上与13个主流离线与在线视频大模型对比。尽管结构简单，SimpleStream表现稳定。仅用4帧时，在OVO-Bench上平均准确率达67.7%，StreamingBench上达80.59%。受控消融实验进一步表明，更长上下文的价值取决于骨干网络，并非随模型规模持续提升；且存在一致的感知-记忆权衡：增加历史上下文可提升回忆能力，但常削弱实时感知。因此，更强的记忆、检索或压缩模块不应被视为进步证据，除非其在相同协议下显著优于SimpleStream。我们主张未来流式基准应分离近期场景感知与长程记忆任务，以更清晰评估复杂性带来的性能提升。
原文摘要 · Abstract (English)
Recent streaming video understanding methods increasingly rely on complex memory mechanisms to handle long video streams. We challenge this trend with a simple finding: a sliding-window baseline that feeds only the most recent N frames to an off-the-shelf VLM already matches or surpasses published streaming models. We formalize this baseline as SimpleStream and evaluate it against 13 major offline and online video LLM baselines on OVO-Bench and StreamingBench. Despite its simplicity, SimpleStream delivers consistently strong performance. With only 4 recent frames, it reaches 67.7% average accuracy on OVO-Bench and 80.59% on StreamingBench. Controlled ablations further show that the value of longer context is backbone-dependent rather than uniformly increasing with model scale, and reveal a consistent perception-memory trade-off: adding more historical context can improve recall, but often weakens real-time perception. This suggests that stronger memory, retrieval, or compression modules should not be taken as evidence of progress unless they clearly outperform SimpleStream under the same protocol. We therefore argue that future streaming benchmarks should separate recent-scene perception from long-range memory, so that performance improvements from added complexity can be evaluated more clearly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。