用VLM当老师，让视频模型在线优化推理能力。
VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
- 让VLM在测试时充当教师，动态生成可微奖励指导视频模型
- 在两个基准上平均提升16.7分，显著超越现有方法
- 适合需要强推理与灵活适应的视频理解任务
近期的「视频推理」范式利用视频生成模型（VGM）生成时间连贯的视觉轨迹以完成推理任务。尽管顶级VGM在视觉质量上表现优异，却常难以理解并遵循特定任务规则，导致各类推理场景中出现逻辑错误。现有方法尝试用视觉语言模型（VLM）作为问题预求解器，生成或优化文本指引供VGM使用，但文本描述难以捕捉复杂的时空细节，即使有合理计划，VGM仍难精准执行细粒度或长尾指令。虽然VLM自身不适合作为求解器，但其具备强大的感知能力，可评估过程约束满足度与最终目标达成情况。基于此，我们提出范式转变：将VLM的角色转为「教师」。具体而言，一个VLM教师提取任务规则，构建可微奖励函数，通过测试时在线优化轻量级LoRA模块，引导视频推理模型。该策略实现自适应测试时优化，扩展了VGM固有的推理边界。在符号型（VBVR-Bench）与通用型（RULER-Bench）视频推理基准上的评估显示，所提方法平均性能提升16.7分，显著优于VLM作求解器（+0.4分）与Best-of-N扩展（+2.2分），且测试成本相当。这些发现表明，将VLM作为测试时教师，是实现可泛化视频推理的有力路径。
原文摘要 · Abstract (English)
The recent "Reasoning with Video" paradigm utilizes Video Generation Models (VGMs) to generate temporally coherent visual trajectories to complete reasoning tasks. Although state-of-the-art VGMs excel at visual quality, they often struggle to understand and follow task-specific rules, leading to logical failures across diverse reasoning scenarios. Existing efforts try to utilize Vision-Language Models (VLMs) as problem pre-solvers to produce or refine textual guidance for the VGM. However, textual descriptions fail to capture intricate spatiotemporal details, and VGMs often struggle to faithfully execute fine-grained or long-tail instructions even with a valid plan. While VLMs struggle as solvers, they possess strong perception capabilities to evaluate process-constraint satisfaction and final-goal achievement. Leveraging this strength, we introduce a paradigm shift that transitions the role of VLMs to "teachers". Specifically, a VLM teacher extracts task-specific rules to formulate differentiable rewards, guiding a VGM Reasoner via test-time online optimization of a lightweight LoRA module. This strategy enables adaptive test-time optimization and extends the reasoning capabilities beyond the VGM's intrinsic boundaries. Evaluations on symbolic (VBVR-Bench) and general-purpose (RULER-Bench) video reasoning benchmarks show that the proposed method yields a 16.7-point average performance gain, outperforming the VLM-as-Solver paradigm (+0.4 points) and Best-of-N scaling (+2.2 points) by a large margin at comparable test-time cost. These findings reveal that integrating VLMs as test-time teachers offers a promising paradigm for achieving generalizable video reasoning. Project Page: https://VLM-as-Teacher.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。