让视频模型像人一样分步推理时间定位，精准找到事件发生时段。
EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization
- 以事件为中心拆解视频，用精简标记高效定位关键片段。
- 在ActivityNet-RTL上达到当前最好效果，视觉标记量大幅减少。
- 零样本下也能在ReXTime任务中表现优异，适合需要可解释推理的场景。
推理性时间定位（RTL）要求模型生成的答案本身包含支持其判断的时间区间，将高层推理与时间定位结合于单一响应中。为此，我们提出首个以事件为中心的视频思维链框架——EventCoT。该框架首先进行事件中心的标记化，将视频转换为紧凑的事件标记，从而高效识别与问题相关的内容。随后在这些事件内进行推理，并通过嵌入匹配将占位标记与视觉嵌入对齐，实现时间区间的精确定位。EventCoT在ActivityNet-RTL上取得当前最优性能，同时使用的视觉标记数量显著少于以往方法，并在地面视频问答基准ReXTime上展现出强大的零样本能力。代码将用于学术研究。
原文摘要 · Abstract (English)
Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, coupling high-level reasoning with temporal grounding in a single response. To tackle this challenge, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization, converting the video into compact event tokens that enable efficient identification of question-relevant events. It then reasons within these events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL while using substantially fewer visual tokens than previous work, and attains strong zero-shot results on the grounded video question answering benchmark ReXTime. Our code will be released for research purposes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。