arXiv:2510.17045cs.CVcs.AI2025-10被引 1

不训练也不用强化学习,用熵信号优化视频推理过程。

Video Reasoning without Training

  • 用输出分布熵作为信号,动态调节模型思考行为。
  • 在多个视频推理数据集上接近强化学习效果,平均差0.6%准确率。
  • 无需训练,比强化学习少用58.6%的token,适合高效推理场景。

基于大视觉语言模型(LMM)的视频推理依赖昂贵的强化学习和冗长的思维链,导致训练与推理阶段计算开销巨大。本文通过分析模型输出分布的熵,发现高质量模型存在微探索-微利用循环,并伴随后期熵峰(更长思考)与最终熵降低,表明其在探索中保持专注、收敛时更具信心。基于此理论发现,我们提出V-Reason(Video-Reason),一种推理时优化方法,通过轻量可训练控制器调整模型值缓存,以熵为基础目标直接调控推理行为,无需强化学习或监督微调。实验表明,V-Reason显著优于基础指令微调模型,在多个视频推理数据集上将与强化学习模型的差距缩小至平均0.6%准确率以内,且无需训练,仅使用58.6%的令牌数,效率大幅提升。

原文摘要 · Abstract (English)

Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, we use the entropy of the model's output distribution as a signal to study and guide reasoning behavior. We discover that high-quality models exhibit a characteristic pattern of micro-exploration and micro-exploitation cycles, followed by a later entropy peak (i.e., longer thinking) and a lower final entropy, indicating more deliberate exploration and confident convergence (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We then use these novel, theoretically-grounded insights to introduce V-Reason (Video-Reason), an inference-time optimization method that adapts the value cache of the LMM through a lightweight, trainable controller. Our proposed controller is guided by an entropy-based objective, to tune the model's behavior directly at inference, without using any RL or supervised fine-tuning. Our experiments show that V-Reason significantly outperforms the base instruction-tuned models on many video reasoning datasets, narrowing the gap with RL models to within 0.6% accuracy on average. We achieve this without any training, while offering efficiency benefits: V-Reason uses 58.6% fewer tokens than the RL model. Project Page https://deepaksridhar.github.io/vreason.github.io/

视频推理推理优化无训练熵控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。