arXiv:2608.28192cs.CV2026-08

提出并行视频定位方法,显著提升生成效率与准确性。

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding

论文配图:Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding
图 1 · 摘自论文原文
  • 采用并行解码机制,打破时间依赖性,固定解码轮次为1+1
  • 在VidSTG上降低79倍延迟,空间解码速度提升92倍
  • 支持零样本迁移,适用于视频问答、目标追踪等任务

时空视频定位(STVG)要求模型识别参考事件发生的时间段,并定位该实体在整个时间段内的位置。现有多模态大语言模型通常以自回归方式串行生成密集的定位轨迹,导致解码延迟随管状轨迹长度增加,且定位误差在时间上传播。本文提出并行管解码(PTD),将定位分解为一个时间块和多个时间条件的空间块,同时解码,消除逐标记和轨迹级依赖,将解码深度固定为1+1轮,与管长无关。为支持并行空间生成,引入解耦块注意力,保留共享视频-查询上下文访问,同时消除跨框依赖,并采用定位感知策略优化时间边界与空间几何。在VidSTG上,PTD相比标准自回归解码降低79倍管完成延迟,空间解码吞吐量提升92倍,同时提升定位准确率。使用紧凑4B主干网络,在VidSTG和HC-STVG上表现优异,并实现零样本迁移至时间定位、视觉问答及指代视频目标跟踪。结果表明,并行管生成是视频中自回归定位的有效替代方案。

原文摘要 · Abstract (English)

Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval. Existing multimodal large language models typically serialize dense localization trajectories autoregressively, causing decoding latency to grow with tube length and allowing localization errors to propagate across time. We introduce Parallel Tube Decoding (PTD), a generative formulation that decomposes grounding into a temporal block followed by time-conditioned spatial blocks decoded simultaneously. This removes both token-level and trajectory-level dependencies, reducing the sequential decoding depth to a fixed $1 + 1$ rounds, independent of tube length. To enable parallel spatial generation, we introduce Decoupled Block Attention, which preserves access to shared video-query context while eliminating cross-box dependencies, together with localization-aware policy optimization for temporal boundaries and spatial geometry. On VidSTG, PTD reduces Tube Completion Latency by 79x and increases spatial decoding throughput by 92x over standard autoregressive decoding, while also improving grounding accuracy. With a compact 4B backbone, our model performs favorably well on VidSTG and HC-STVG, and generalizes zero-shot to temporal grounding, grounded VideoQA, and referring video object tracking. Our results show parallel tube generation is an efficient and effective alternative to autoregressive localization in videos.

视频定位并行解码高效生成多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。