提出一种无需训练的视频时间定位方法，解决语义碎片化问题。
TAG: A Simple Yet Effective Temporal-Aware Approach for Zero-Shot Video Temporal Grounding
- 引入时序池化与一致性聚类，捕捉视频时序上下文。
- 在Charades-STA和ActivityNet上达到最新性能，无需LLM推理。
- 方法简洁高效，适合零样本视频定位场景应用。
视频时间定位（VTG）旨在根据自然语言查询提取相关视频片段。近期的零样本方法利用预训练视觉语言模型（VLMs）实现无需额外训练的目标时刻定位，但存在语义碎片化问题：时序连续且语义相同的帧被分割到多个片段中，导致难以准确预测与文本一致的目标时刻。同时，现有方法依赖扭曲的相似度分布进行定位，且严重依赖需高成本推理的LLMs。为此，本文提出一种简单而有效的时序感知方法TAG，包含时序池化、时序一致性聚类与相似度调整。该方法无需训练即可有效捕捉视频时序上下文，缓解相似度分布失真问题。在Charades-STA和ActivityNet Captions数据集上取得当前最优结果，且不依赖LLM。代码已开源。
原文摘要 · Abstract (English)
Video Temporal Grounding (VTG) aims to extract relevant video segments based on a given natural language query. Recently, zero-shot VTG methods have gained attention by leveraging pretrained vision-language models (VLMs) to localize target moments without additional training. However, existing approaches suffer from semantic fragmentation, where temporally continuous frames sharing the same semantics are split across multiple segments. When segments are fragmented, it becomes difficult to predict an accurate target moment that aligns with the text query. Also, they rely on skewed similarity distributions for localization, making it difficult to select the optimal segment. Furthermore, they heavily depend on the use of LLMs which require expensive inferences. To address these limitations, we propose a \textit{TAG}, a simple yet effective Temporal-Aware approach for zero-shot video temporal Grounding, which incorporates temporal pooling, temporal coherence clustering, and similarity adjustment. Our proposed method effectively captures the temporal context of videos and addresses distorted similarity distributions without training. Our approach achieves state-of-the-art results on Charades-STA and ActivityNet Captions benchmark datasets without rely on LLMs. Our code is available at https://github.com/Nuetee/TAG
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。