通过关键标记强化视频推理,提升模型准确率与可解释性。
Video-KTR: Reinforcing Video Reasoning via Key Token Attribution
- 基于视觉、时间、熵值三类信号筛选关键标记,进行细粒度强化学习。
- 在Video-Holmes上达到42.7%准确率,超越GPT-4o。
- 适用于需要高精度视频理解与可解释性的研究者与开发者。
强化学习(RL)在提升多模态大语言模型推理能力方面展现出强大潜力,但现有视频推理方法常依赖粗粒度序列级奖励或单一因子标记选择,忽视了视觉输入、时间动态与语言输出之间的细粒度关联,限制了准确性与可解释性。本文提出Video-KTR,一种模态感知的策略塑造框架,通过结合三种归因信号实现选择性、标记级强化学习:(1)通过反事实掩码识别视觉敏感标记,揭示感知依赖;(2)通过帧乱序检测时间敏感标记,暴露时间敏感性;(3)通过高熵标记标识预测不确定性。仅对这些关键标记进行强化,使Video-KTR聚焦于语义信息丰富、模态敏感的内容,同时过滤低价值标记。在五个挑战性基准上,Video-KTR达到最先进或极具竞争力的结果,在Video-Holmes上达42.7%准确率,超越GPT-4o,且在推理与通用视频理解任务中均保持一致提升。消融实验验证了各归因信号的互补作用及目标标记更新的鲁棒性。整体上,Video-KTR提升了准确率与可解释性,为复杂视频推理提供简单、即插即用的强化学习扩展。代码与模型已开源。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has shown strong potential for enhancing reasoning in multimodal large language models, yet existing video reasoning methods often rely on coarse sequence-level rewards or single-factor token selection, neglecting fine-grained links among visual inputs, temporal dynamics, and linguistic outputs, limiting both accuracy and interpretability. We propose Video-KTR, a modality-aware policy shaping framework that performs selective, token-level RL by combining three attribution signals: (1) visual-aware tokens identified via counterfactual masking to reveal perceptual dependence; (2) temporal-aware tokens detected through frame shuffling to expose temporal sensitivity; and (3) high-entropy tokens signaling predictive uncertainty. By reinforcing only these key tokens, Video-KTR focuses learning on semantically informative, modality-sensitive content while filtering out low-value tokens. Across five challenging benchmarks, Video-KTR achieves state-of-the-art or highly competitive results, achieving 42.7\% on Video-Holmes (surpassing GPT-4o) with consistent gains on both reasoning and general video understanding tasks. Ablation studies verify the complementary roles of the attribution signals and the robustness of targeted token-level updates. Overall, Video-KTR improves accuracy and interpretability, offering a simple, drop-in extension to RL for complex video reasoning. Our code and models are available at https://github.com/zywang0104/Video-KTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。