arXiv:2506.10821cs.CVcs.AI2025-06被引 22

让视频模型像人一样一步步思考,看懂长视频

VideoExplorer: Think With Videos For Agentic Long-Video Understanding

  • 通过迭代提问、定位关键片段,逐步推理长视频内容
  • 在多个基准上超越现有方法,准确率显著提升
  • 适合需要深度理解长视频的科研与应用开发

长视频理解(LVU)是计算机视觉中的难题。现有方法要么降采样帧进行单次推理,损失细节;要么依赖通用文本推理,缺乏任务感知。本文提出VideoExplorer框架,基于“用视频思考”的原则,将规划、时间定位与可扩展感知融为一体。它不依赖静态上下文,而是迭代生成子问题、定位相关片段,并执行面向任务的时序可扩展理解,直至得出答案,实现忠实、高效且可解释的推理。为解决训练资源匮乏问题,我们采用难度自适应采样构建长视频推理数据集,确保复杂任务下高质量轨迹。在此基础上,设计两阶段训练:监督轨迹初始化后,进行轨迹级偏好优化,引导由下游奖励驱动的自适应时间定位与信息融合。在主流长视频理解与推理基准上的大量实验表明,VideoExplorer显著优于现有基线,展现出强鲁棒性、自适应性与效率。代码已公开于https://github.com/yhy-2000/VideoDeepResearch。

原文摘要 · Abstract (English)

Long-video understanding~(LVU) is a challenging problem in computer vision. Existing methods either downsample frames for single-pass reasoning, sacrificing fine-grained details, or depend on textual reasoning over task-agnostic representations, hindering task-specific perception and exploration. In this paper, we propose VideoExplorer, a framework grounded in the principle of ``thinking with video'', which naturally intertwines planning, temporal grounding, and scalable perception into a coherent reasoning process. Rather than reasoning over a static context, VideoExplorer iteratively formulates sub-questions, locates relevant moments, and performs task-oriented, temporally scalable video understanding until reaching the final answer, enabling faithful, efficient, and interpretable reasoning. To address the lack of LVU training resources, we construct a long-video reasoning dataset using difficulty-adaptive sampling to ensure high-quality trajectories on complex tasks. Building on this dataset, we design a two-stage training pipeline: supervised trajectory initialization followed by trajectory-level preference optimization, encouraging adaptive temporal grounding and iterative information integration guided by downstream rewards. Extensive evaluations on popular long-video understanding and reasoning benchmarks demonstrate VideoExplorer's significant advantage over existing baselines, highlighting its robustness, adaptability, and efficiency. Our code is made publicly available in this repository(https://github.com/yhy-2000/VideoDeepResearch).

长视频理解思维链视频推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。