用大模型当老师,给小模型补关键视频片段,提升推理准确率。
Find, Fix, Reason: Context Repair for Video Reasoning

- 大模型作为教师,动态提供缺失的时空证据片段。
- 在多个基准上提升准确率,且泛化能力强。
- 适合需要高效视频推理的场景,如智能监控、教育分析。
强化学习推动了大模型在视频推理中的发展,但主流方法要么依赖于策略自探索(易达知识边界),要么采用混合回放需精细正则化。动态上下文方法虽聚焦关键证据,但常需预训练和两阶段调优,且受限于小模型能力。相比之下,大模型在指令遵循与多模态理解方面表现优异,能为小模型提供更丰富的上下文,并通过简单工具快速定位目标区域。基于此,我们提出一种观察级干预:冻结的、工具集成的教师识别缺失的时空依赖关系,从原始视频中提供最小证据补丁(如时间戳、区域等),问题保持不变。学生模型在加入上下文后重新作答,训练采用融入分组相对策略优化(GRPO)的选定回合方案。我们进一步提出鲁棒改进奖励(RIR),兼顾答案正确性与推理过程对证据的引用一致性。该方法在批次内归一化,保留在线探索能力,同时引导其沿因果有意义方向进行微小调整。在多个相关基准测试中,均实现一致的准确率提升与强泛化性能。代码将发布于 https://jethrojames.github.io/FFR/。
原文摘要 · Abstract (English)
Reinforcement learning has advanced video reasoning in large multi-modal models, yet dominant pipelines either rely on on-policy self-exploration, which plateaus at the model's knowledge boundary, or hybrid replay that mixes policies and demands careful regularization. Dynamic context methods zoom into focused evidence but often require curated pretraining and two-stage tuning, and their context remains bounded by a small model's capability. In contrast, larger models excel at instruction following and multi-modal understanding, can supply richer context to smaller models, and rapidly zoom in on target regions via simple tools. Building on this capability, we introduce an observation-level intervention: a frozen, tool-integrated teacher identifies the missing spatiotemporal dependency and provides a minimal evidence patch (e.g., timestamps, regions etc.) from the original video while the question remains unchanged. The student answers again with the added context, and training updates with a chosen-rollout scheme integrated into Group Relative Policy Optimization (GRPO). We further propose a Robust Improvement Reward (RIR) that aligns optimization with two goals: outcome validity through correct answers and dependency alignment through rationales that reflect the cited evidence. Advantages are group-normalized across the batch, preserving on-policy exploration while directing it along causally meaningful directions with minimal changes to the training stack. Experiments on various related benchmarks show consistent accuracy gains and strong generalization. Code will be available at https://jethrojames.github.io/FFR/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。