arXiv:2602.13602cs.CVcs.LG2026-02中稿 · CVPR被引 1

用稀疏帧推理提升视频问答效率与准确率

Towards Sparse Video Understanding and Reasoning

  • 按需选关键帧,用摘要状态跨轮次传递信息
  • 在多个数据集上减少90%以上帧数,准确率不降反升
  • 适合资源受限场景,支持主流视觉语言模型

我们提出 evise(基于视频稀疏性的推理),一种多轮交互式视频问答智能体。不同于均匀采样帧, evise 动态选取少量关键帧,维护摘要作为状态,并在有把握时提前停止。它可在“即插即用”模式下适配私有视觉语言模型(VLMs),并支持开源模型的强化学习微调。针对微调,我们设计 EAGER(证据调整收益),一种无需标注的奖励机制,包含三项:(1)置信度增益——新增帧后,正确选项与最强干扰项的对数几率差增大则获奖励;(2)摘要充分性——仅用最后提交的摘要重问问题并成功回答则奖励;(3)正确且早停——在小轮次预算内正确回答获得奖励。在多个视频问答基准测试中, evise 显著降低帧数、轮数和提示词数量,同时提升准确率,验证了实用的稀疏视频推理能力。

原文摘要 · Abstract (English)

We present \revise (\underline{Re}asoning with \underline{Vi}deo \underline{S}parsity), a multi-round agent for video question answering (VQA). Instead of uniformly sampling frames, \revise selects a small set of informative frames, maintains a summary-as-state across rounds, and stops early when confident. It supports proprietary vision-language models (VLMs) in a ``plug-and-play'' setting and enables reinforcement fine-tuning for open-source models. For fine-tuning, we introduce EAGER (Evidence-Adjusted Gain for Efficient Reasoning), an annotation-free reward with three terms: (1) Confidence gain: after new frames are added, we reward the increase in the log-odds gap between the correct option and the strongest alternative; (2) Summary sufficiency: at answer time we re-ask using only the last committed summary and reward success; (3) Correct-and-early stop: answering correctly within a small turn budget is rewarded. Across multiple VQA benchmarks, \revise improves accuracy while reducing frames, rounds, and prompt tokens, demonstrating practical sparse video reasoning.

视频理解稀疏推理强化学习多轮问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。