arXiv:2607.02959cs.CVcs.LG2026-07被引 1

让视觉语言模型主动搜视频找答案,提升长视频问答准确率。

Incentivizing Vision Language Models to Search for Long Video Question Answering

论文配图:Incentivizing Vision Language Models to Search for Long Video Question Answering
图 1 · 摘自论文原文
  • 用自然语言驱动搜索,分步定位长视频中的关键片段。
  • 通过强化学习优化搜索与推理,使检索结果准确率提升8%~15%。
  • 适合关注长视频理解、智能搜索与多阶段推理的研究者。

我们提出VSeek,一种智能体框架,将长视频问答(LVQA)从被动的单次感知任务转变为多轮检索过程。VSeek利用自然语言驱动的搜索,在长视频中定位相关上下文,并通过强化学习(RL)进行后训练,联合优化精准搜索查询与检索片段的推理能力。尽管强化学习在数学和代码等符号领域已取得突破,但在长视频理解中的应用受限于缺乏可验证的奖励信号。为此,我们提出一种新颖的神经符号方法,将开放式的自然语言问题转化为形式化的时间逻辑规范,系统性地分解问题为一系列原子视觉要素(如关键对象、行为)及其时序关系。这些结构化的语义事件作为强化学习的反馈信号,提供密集且可验证的奖励,而非仅依赖最终答案正确性。通过显式优化证据搜索行为,VSeek在基准测试中将Pass@1分数提升最多8%,Pass@4分数提升15%,显著优于基础模型。代码已开源:https://utaustin-swarmlab.github.io/VSeek。

原文摘要 · Abstract (English)

We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process. VSeek utilizes a natural language-driven search to identify relevant context within long videos and is post-trained with reinforcement learning (RL) to jointly formulate targeted search queries and reason over retrieved clips for LVQA. While RL post-training has revolutionized reasoning in symbolic domains such as mathematics and code, its application to long-video understanding remains hindered by a lack of verified rewards. To ensure that the retrieved context is relevant, we propose a novel neuro-symbolic approach that bridges open-ended natural language with discrete visual verification. Specifically, complex user queries are compiled into formal temporal logic specifications for systematically decomposing natural language questions into a definitive checklist of required atomic visual primitives, such as key objects and activities, along with their temporal ordering. These systematically derived grounding events provide the critical feedback signal for RL post-training, enabling dense, verifiable rewards based on the successful retrieval of these specific visual elements rather than relying entirely on outcome-only answer accuracy. By explicitly optimizing for this verifiable evidence-seeking behavior, VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks compared to base models. We open-source our code at https://utaustin-swarmlab.github.io/VSeek.

长视频问答强化学习视觉推理智能搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。