用强化学习让模型自适应搜索长视频关键帧,提升理解准确率。
TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learning
- 将视频搜索融入推理过程,通过自验证机制优化搜索策略。
- 在多个长视频基准上超越现有模型,最高提升4.1%。
- 适合需要精准定位长视频内容的场景,如智能检索与分析。
时间搜索旨在从数万帧视频中识别出最相关的最少帧数,是实现长视频精准理解的基础。现有方法通常依赖人工设计的搜索流程,缺乏端到端优化以学习最优搜索策略。本文提出TimeSearch-R,将时间搜索重构为文本-视频交织的思维过程,通过强化学习(RL)将搜索视频片段无缝融入推理流程。然而,应用如分组相对策略优化(GRPO)等RL训练方法于视频推理时,可能导致中间搜索决策无监督,造成对视频内容探索不足及逻辑推理不一致。为此,我们引入带有完整性自验证的GRPO(GRPO-CSV),从交织推理过程中收集已搜索帧,并使用同一策略模型验证其充分性,从而提升视频推理的完整性。此外,我们构建了专用于SFT冷启动和RL训练的高质量数据集,剔除弱时间依赖样本以增强任务难度并提升搜索能力。大量实验表明,TimeSearch-R在Haystack-LVBench、Haystack-Ego4D等时间搜索基准,以及VideoMME、MLVU等长视频理解基准上均取得显著提升。尤其在LongVideoBench上,相比基线模型Qwen2.5-VL提升4.1%,较先进视频推理模型Video-R1提升2.0%。代码已开源。
原文摘要 · Abstract (English)
Temporal search aims to identify a minimal set of relevant frames from tens of thousands based on a given query, serving as a foundation for accurate long-form video understanding. Existing works attempt to progressively narrow the search space. However, these approaches typically rely on a hand-crafted search process, lacking end-to-end optimization for learning optimal search strategies. In this paper, we propose TimeSearch-R, which reformulates temporal search as interleaved text-video thinking, seamlessly integrating searching video clips into the reasoning process through reinforcement learning (RL). However, applying RL training methods, such as Group Relative Policy Optimization (GRPO), to video reasoning can result in unsupervised intermediate search decisions. This leads to insufficient exploration of the video content and inconsistent logical reasoning. To address these issues, we introduce GRPO with Completeness Self-Verification (GRPO-CSV), which gathers searched video frames from the interleaved reasoning process and utilizes the same policy model to verify the adequacy of searched frames, thereby improving the completeness of video reasoning. Additionally, we construct datasets specifically designed for the SFT cold-start and RL training of GRPO-CSV, filtering out samples with weak temporal dependencies to enhance task difficulty and improve temporal search capabilities. Extensive experiments demonstrate that TimeSearch-R achieves significant improvements on temporal search benchmarks such as Haystack-LVBench and Haystack-Ego4D, as well as long-form video understanding benchmarks like VideoMME and MLVU. Notably, TimeSearch-R establishes a new state-of-the-art on LongVideoBench with 4.1% improvement over the base model Qwen2.5-VL and 2.0% over the advanced video reasoning model Video-R1. Our code is available at https://github.com/Time-Search/TimeSearch-R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。