arXiv:2508.04546cs.CV2025-08ICCV被引 8

提出分层事件记忆机制,实现低延迟高精度在线视频时间定位。

Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Grounding

  • 基于事件提议建模不同持续时长的事件信息
  • 分层记忆保留近期与长期历史事件,提升定位准确率
  • 新增未来预测分支,支持实时推理,适合流式视频应用

本文针对在线视频时间定位(OnVTG)任务,要求模型在不预览未来帧的情况下,从持续输入的视频流中定位与文本查询相关的事件。由于在线视频无限流式输入,存储全部历史数据不现实。现有方法虽使用记忆模块保存近期帧特征,但缺乏有效事件建模且难以保留长期信息,导致性能受限。为此,我们提出一种分层事件记忆框架:通过事件提议捕捉多时长事件信息,并构建分层记忆结构以同时保留近期与长期历史事件;此外,引入未来预测分支,提前判断目标事件是否即将发生并回归起始时间。该方法在TACoS、ActivityNet Captions和MAD数据集上达到当前最优性能。代码已开源。

原文摘要 · Abstract (English)

In this paper, we tackle the task of online video temporal grounding (OnVTG), which requires the model to locate events related to a given text query within a video stream. Unlike regular video temporal grounding, OnVTG requires the model to make predictions without observing future frames. As online videos are streaming inputs and can go on indefinitely, it is impractical and inefficient to store all historical inputs. The existing OnVTG models employ memory to store recent historical video frame features and predict scores indicating whether the current frame corresponds to the start or end time of the target event. However, these methods lack effective event modeling and cannot retain long-term historical information, leading to low performance. To tackle these challenges, we propose a hierarchical event memory for OnVTG. We propose an event-based OnVTG framework that makes predictions based on event proposals that model event-level information with various durations. To preserve historically valuable event information, we introduce a hierarchical event memory that retains historical events, allowing the model to access both recent and long-term information. To enable the real-time prediction, we further propose a future prediction branch that predicts whether the target event will occur shortly and further regresses the start time of the event. We achieve state-of-the-art performance on the TACoS, ActivityNet Captions, and MAD datasets. Code is available at https://github.com/minghangz/OnVTG.

视频定位在线推理事件记忆流式处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。