arXiv:2512.14273cs.CV2025-12被引 6

通过分阶段放大关键帧,提升视频问答的定位精度与答案可信度。

Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-in

  • 先粗后精定位视频片段,聚焦关键帧进行细粒度验证。
  • 在NExT-GQA和ReXTime上提升定位准确率5.2%和4.6%。
  • 适合长视频理解与需要高可信答案的场景。

基于视觉语言模型的视频问答任务旨在定位视频中的相关时间片段并生成准确答案,但现有大模型普遍存在时间感知不足的问题。尽管已有基于组相对策略优化(GRPO)的方法尝试改进时间定位,仍难以忠实依据视频证据,导致定位偏差和幻觉。本文提出Zoom-Zero框架,采用粗到精策略:先定位与问题相关的片段,再对最显著帧进行时间放大,实现更细粒度的视觉验证。核心创新包括:(i) 放大精度奖励机制,验证定位结果的准确性并支持对定位帧的细粒度视觉分析;(ii) 令牌选择性信用分配,将奖励精准分配给负责时间定位或答案生成的令牌,缓解GRPO在多维度奖励信号下的失效问题。实验表明,该方法在NExT-GQA和ReXTime上分别提升时间定位准确率5.2%和4.6%,平均答案准确率提升2.4%。推理阶段的分阶段放大策略还显著改善长视频理解,在长视频基准上平均提升6.4%,同时保留关键视觉细节且不丢失全局上下文。

原文摘要 · Abstract (English)

Grounded video question answering (GVQA) aims to localize relevant temporal segments in videos and generate accurate answers to a given question; however, large video-language models (LVLMs) exhibit limited temporal awareness. Although existing approaches based on Group Relative Policy Optimization (GRPO) attempt to improve temporal grounding, they still struggle to faithfully ground their answers in the relevant video evidence, leading to temporal mislocalization and hallucinations. In this work, we present Zoom-Zero, a coarse-to-fine framework that first localizes query-relevant segments and then temporally zooms into the most salient frames for finer-grained visual verification. Our method addresses the limits of GRPO for the GVQA task with two key innovations: (i) a zoom-in accuracy reward that validates the fidelity of temporal grounding prediction and facilitates fine-grained visual verification on grounded frames; (ii) token-selective credit assignment, which attributes rewards to the tokens responsible for temporal localization or answer generation, mitigating GRPO's issue in handling multi-faceted reward signals. Our proposed method advances grounded video question answering, improving temporal grounding by 5.2\% on NExT-GQA and 4.6\% on ReXTime, while also enhancing average answer accuracy by 2.4\%. Additionally, the coarse-to-fine zoom-in during inference further benefits long-form video understanding by preserving critical visual details without compromising global context, yielding an average improvement of 6.4\% on long-video benchmarks.

视频问答时间定位强化学习长视频理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。