arXiv:2608.25356cs.CV2026-08

聚焦视频中的关键片段,提升长视频理解准确率。

Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding

论文配图:Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
图 1 · 摘自论文原文
  • 训练时用关键片段引导模型,仅需少量帧输入。
  • 在多个基准上优于基础模型,且不依赖标注答案。
  • 适合需要高效处理长视频的场景,如视频问答。

视觉语言模型在长视频理解任务中取得显著进展,通常基于全视频采样帧进行问答。但随着视频变长,全视频上下文包含更多与问题无关的内容,干扰模型判断。我们发现,仅聚焦于标注的关键证据片段,相比使用完整视频,能显著提升不同规模模型的预测准确率,同时减少输入帧数。基于此,提出Clue-OPSD:一种以关键片段为特权监督的在线自蒸馏框架。训练时,全视频学生模型通过与由关键片段条件化的自教师对齐下一词分布,在学生生成轨迹上实现知识迁移。该方法无需真实答案标签或额外推理模块,即可实现持续性能提升。在多个长视频理解基准及Qwen3.5系列模型上验证,均显著优于基线模型,并达到监督后训练基线水平。

原文摘要 · Abstract (English)

Vision-language models (VLMs) have made substantial progress in long-video understanding, with standard backbone models typically answering questions from frames sampled across the full video. However, as videos become longer, the full-video context inevitably contains more question-irrelevant temporal content, which can distract the model from the evidence needed to answer a specific question. We empirically find that focusing the visual input on short annotated clue intervals containing question-relevant evidence consistently improves prediction accuracy across model scales compared with using the corresponding full videos, while requiring fewer input frames. Based on this finding, we introduce Clue-OPSD, a clue-privileged on-policy self-distillation framework for long-video understanding. During training, a full-video student learns from a self-teacher conditioned on the corresponding clue interval by aligning their next-token distributions along student-generated trajectories. Clue-OPSD thus uses clue intervals as privileged supervision without relying on ground-truth answer labels, while requiring no clue annotations or additional modules at inference time. Extensive experiments across multiple long-video understanding benchmarks and Qwen3.5 model scales demonstrate consistent improvements over the corresponding backbone models and strong performance against supervised post-training baselines.

长视频理解自蒸馏视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。