用分段推理提升长视频问答效率，节省50%帧数还能更准。
Video-in-the-Loop: Span-Grounded Long Video QA with Interleaved Reasoning
- 先低帧率定位关键片段，再高帧率重分配视觉令牌回答问题
- 在固定令牌预算下，比基线高出8.6%准确率，帧数减少50%
- 支持直接追溯答案来源，适合需要可解释性的长视频分析
我们提出视频回路（ViTL），一种两阶段长视频问答框架。通过低帧率预览定位问题相关片段，再以更高有效帧率进行基于片段的视觉令牌重分配来回答问题，输出包含片段与最终选项的交错结果，实现直接溯源。我们还引入 extit{dataset}，将基于描述的事件图转化为带真实时间跨度的多选题问答，每道题配以真实时间跨度和推理链。ViTL 采用端到端训练，使用交错组相对目标函数，同时优化定位的时序交并比与答案正确性，使答案反馈能反向指导片段定位，无需额外计算开销。在固定令牌预算下，ViTL 在长视频问答与时间定位任务（如 Charades-STA、ActivityNet-Captions）中最高提升 8.6%，且帧输入减少 50%。消融实验表明，基于片段的令牌重分配优于均匀采样。 extit{dataset} 与 ViTL 共同提供一种可解释、高效、可扩展的长视频问答方案。
原文摘要 · Abstract (English)
We present \emph{Video-in-the-Loop} (ViTL), a two-stage long-video QA framework that preserves a fixed token budget by first \emph{localizing} question-relevant interval(s) with a low-fps skim and then \emph{answering} via span-aware reallocation of visual tokens at higher effective frame rate, emitting an interleaved output with both spans and the final option for direct attribution. We also introduce \dataname{}, which converts description based event graphs into \emph{span-grounded} multiple-choice QA by pairing each question with \emph{ground-truth} time span(s) and related reasoning. ViTL is trained end-to-end with an interleaved group-relative objective that couples temporal IoU for localization with answer correctness, allowing credit to flow from answers back to spans without increasing compute. Under fixed token budgets, ViTL attains up to 8.6% with 50% less frame input on long-video QA and temporal grounding (e.g., Charades-STA, ActivityNet-Captions) and ablations show that span-aware token reallocation consistently surpasses uniform sampling. Together, \dataname{} and ViTL provide an interpretable, compute-efficient recipe for scalable long-video QA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。