arXiv:2608.16316cs.CVcs.AI2026-08被引 1

让小模型学会大模型的视频推理能力,关键在捕捉中间思考过程。

Deep Thought Alignment: Trajectory-Level Latent Distillation for Video Reasoning

论文配图:Deep Thought Alignment: Trajectory-Level Latent Distillation for Video Reasoning
图 1 · 摘自论文原文
  • 在轨迹末端对齐隐状态,让小模型学习大模型的推理过程。
  • 在帧数少、长视频等场景下,性能提升显著超过传统方法。
  • 适合需要高效视频理解的应用,如移动端视频分析。

用于视频推理的大规模多模态模型长期受限于处理海量视觉信息带来的高计算成本。为解决此问题,本文提出将大模型的推理能力迁移到更高效的小模型。现有基于输出策略的蒸馏方法(OPD)仅对生成的词元分布进行对齐,但无法约束推理过程中形成的隐状态。为此,本文提出潜空间-OPD(Latent-OPD),在轨迹末端对齐隐状态,使其有效总结多帧累积的视觉证据与推理上下文。同时引入渐进式教师前瞻策略,使学生模型中后层逐步对齐更深的教师层。在六个视频推理基准上的实验表明,Latent-OPD持续优于仅输出对齐的OPD,尤其在帧数有限、长视频或复杂证据聚合任务中表现突出。结果证明该方法是实现高效视频推理的有效方案。

原文摘要 · Abstract (English)

Large Multimodal Models (LMMs) for video reasoning have long been hindered by the high computational cost of processing vast amounts of visual information. This dilemma motivates the transfer of the reasoning capabilities of large models to smaller, more efficient ones. On-Policy Distillation (OPD) offers a promising solution by matching output-token distributions along student-generated trajectories. However, video reasoning often depends on evidence accumulated across multiple frames. In this context, output-level supervision only captures information expressed through token predictions and does not directly constrain the latent representations formed during reasoning. To address this limitation, we propose Latent-OPD, which augments OPD with trajectory-level latent distillation. Specifically, our method focuses on the position at the end of each trajectory, where hidden states effectively summarize the accumulated visual evidence and reasoning context. Furthermore, we introduce a progressive teacher-lookahead strategy, which aligns middle-to-late student layers with increasingly deeper teacher layers. Experiments on six video reasoning benchmarks show that Latent-OPD consistently outperforms output-only OPD. Notably, the improvements are particularly pronounced in scenarios with limited frames, long videos, or tasks requiring complex evidence aggregation. These results establish Latent-OPD as a highly effective approach to frame-efficient video reasoning.

视频推理模型蒸馏隐状态对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。