让AI从任务需求中定位视频关键片段,更贴近真实应用。
TimeScope: Towards Task-Oriented Temporal Grounding In Long Videos
- 通过逐步推理实现粗到精的时序定位
- 在多任务场景下显著提升定位精度和下游任务表现
- 适合需要理解任务意图的长视频分析场景
识别长视频中的关键时间区间(即时序定位,TG)对视频理解与推理至关重要。本文提出一种新范式——任务导向时序定位(ToTG),其输入由下游任务需求驱动,而非明确的时间区间描述。例如,输入为“解释视频中男子为何被送进医院”,而传统TG需给出如“男子被石头绊倒并摔倒的时刻”这类具体描述。该新范式对现有方法构成挑战,要求同时具备深层任务理解与细粒度时序定位能力。为此,我们构建了首个综合性基准ToTG-Bench,并提出新方法TimeScope,通过渐进式推理实现粗到精定位。该方法基于大量精心设计的思维链(CoT)数据进行监督微调,具备强泛化能力。评估表明,TimeScope在三个维度上均优于现有基线:(1)定位精度显著提升;(2)有效促进下游任务性能;(3)跨任务、跨领域表现稳健。所有模型、数据集与源码将开源,以推动该方向研究。
原文摘要 · Abstract (English)
Identifying key temporal intervals within long videos, known as temporal grounding (TG), is important to video understanding and reasoning tasks. In this paper, we introduce a new form of the temporal grounding problem, \textbf{Task-oriented Temporal Grounding} (\textbf{ToTG}), which is driven by the requirements of downstream tasks rather than explicit time-interval descriptions. For example, a ToTG input may be "explain why the man in the video is sent to the hospital," whereas traditional TG would take an explicit temporal description such as "the moments when the man is tripped by a stone and falls to the ground." This new ToTG formulation presents significant challenges for existing TG methods, as it requires jointly performing deep task comprehension and fine-grained temporal localization within long videos. To address these challenges, we conduct a systematic set of studies. First, we construct \textbf{a new benchmark ToTG-Bench}, which comprehensively evaluates ToTG performance across diverse settings. Second, we introduce \textbf{a new temporal-ground method TimeScope}, which performs coarse-to-fine localization through a progressive reasoning process. Leveraging extensive supervised fine-tuning with carefully curated chain-of-thought (CoT) data from a variety of scenarios, TimeScope generalizes effectively across tasks and domains. Our evaluation demonstrates \textbf{TimeScope's empirical advantages} over existing baselines from three perspectives: (1) substantial improvements in grounding precision, (2) significant benefits to downstream tasks, and (3) strong generalizability across different scenarios. All models, datasets, and source code will be fully open-sourced to support future research in this area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。