用强化学习优化视频帧选择,让大模型更懂上下文。
ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding
- 用策略梯度强化学习动态选帧,根据查询自动优化
- 在多个视频问答数据集上准确率显著提升
- 无需人工标注帧,适合视频理解任务研究者
大型多模态模型(LMMs)在视觉-语言推理上取得进展,但视频理解仍受限于低效的帧选择策略。现有方法依赖静态规则或外部检索模块,难以捕捉与用户查询相关的语义信息。本文提出ReFoCUS(基于强化学习的上下文理解帧优化框架),首次将在线策略梯度强化学习引入视频-语言模型的帧级优化中。该框架通过参考模型生成的奖励信号,学习帧组合的选择策略,以捕获支持时序一致回答的最佳帧组合。采用自回归且查询条件化的选帧结构,在保证上下文一致性的同时降低组合空间复杂度。策略学习无需显式帧级监督,能隐式发现语义一致的最优帧组合。在多个视频问答基准测试中,ReFoCUS持续提升推理准确率,证明了帧选择与模型内部效用对齐的优势。
原文摘要 · Abstract (English)
Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to video understanding remains constrained by suboptimal frame selection strategies, albeit with the rapid development of video-specialized LMMs. Prior works attempted to solve this with static heuristics or external retrieval modules to feed frame-level information, but these approaches often fail to capture visual cues grounded to the given user queries conflating raw visual dynamics with true semantic relevance. In this paper, we introduce ReFoCUS (Reinforcement-guided Frame Optimization for Contextual UnderStanding), the first framework to integrate online policy-gradient reinforcement learning into frame-level optimization for video-LLMs. ReFoCUS aims to learn a frame selection policy, leveraging reward signals derived from reference models to capture their underlying scoring behavior over frame combinations that best support temporally grounded responses. To efficiently explore the large combinatorial frame space, we employ an autoregressive and query-conditional selection architecture that ensures contextual consistency while reducing complexity. Our policy learning removes the need for explicit frame-level supervision, as it implicitly discovers optimal and semantically consistent frame compositions. ReFoCUS consistently improves reasoning accuracy across multiple video QA benchmarks, demonstrating the advantage of aligning frame selection with model-internal utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。