arXiv:2607.28312cs.CVcs.AI2026-07被引 1

用视觉大模型中的潜在物体作记忆锚点,实现高效流式视频理解。

ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding

论文配图:ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
图 1 · 摘自论文原文
  • 将冻结的视频大模型特征转化为空间连贯的潜在物体作为记忆锚点。
  • 在在线评估中提升模型性能10.0分,同时降低50%显存与首帧响应时间。
  • 无需额外检测或分割模型,适合资源受限的实时视频应用。

流式视频理解要求模型持续保留未来问题未知时的有用视觉证据。现有方法主要依据标记重要性、时间冗余或段级相关性管理不断增长的视觉上下文,但很少围绕随时间持续演化的目标组织证据。本文提出ObjectStream,一种无需训练的框架,将潜在物体作为流式视频理解的记忆锚点。ObjectStream直接从冻结的Video-LLM表示中诱导出空间连贯的潜在物体,跨帧关联形成持久锚点,并在有限内存预算下维护其历史,无需外部目标检测器或分割模型。基于这些锚点,ObjectStream保存三种互补证据:持久物体历史、瞬时变化及近期视觉上下文。该设计使现有Video Large Language Models(Video-LLMs)能在不改变底层模型的前提下,推理物体身份、交互与状态变化。在在线流式和离线长视频基准上的实验表明,其兼具有效性与效率。在线评估中,ObjectStream使Qwen2.5-VL-7B在OVO-Bench Real-Time Visual Perception上提升10.0分,同时峰值显存与首字节传输时间减少约50%;在离线长视频基准上,超越全标记基线,同时丢弃82.5%的视觉标记。结果凸显潜在物体作为紧凑流式视频记忆的实用且有效组织原则。

原文摘要 · Abstract (English)

Streaming video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to token importance, temporal redundancy, or segment-level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training-free framework that treats latent objects as memory anchors for streaming video understanding. ObjectStream induces spatially coherent latent objects directly from frozen Video-LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded memory budget, without requiring external object detectors or segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing Video Large Language Models (Video-LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long-video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception, while reducing peak GPU mem-ory and TTFT by approximately 50%. On offline long-video benchmarks, it surpasses the full-token baseline while discarding 82.5% of visual tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming video memory.

视频理解流式处理记忆锚点视觉大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。