arXiv:2601.23224cs.CV2026-01被引 16

让AI像侦探一样在长视频中逐步找线索,精准定位关键片段。

Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning

  • 分步迭代寻找视觉线索,动态停止以节省计算
  • 在MLVU上达72.1%准确率,超越现有方法
  • 适合需要深度视频推理的研究者与开发者

现有用于长视频理解的多模态大模型主要依赖均匀采样和单轮推理,难以在大量冗余信息中发现稀疏但关键的证据。我们提出Video-o3框架,支持迭代式发现显著视觉线索、对关键片段进行细粒度检查,并在获取足够证据后自适应终止。技术上,针对交错调用工具的两个核心挑战:首先,提出任务解耦注意力掩码,缓解推理与工具调用异质性导致的关注分散;其次,引入可验证轨迹引导奖励,平衡探索覆盖率与推理效率。为支持大规模训练,我们构建了包含173K高质量工具交互轨迹的Seeker-173K数据集,支持监督与强化学习。大量实验表明,Video-o3显著优于现有方法,在MLVU上达到72.1%准确率,在Video-Holmes上达46.5%,验证了其强大的多跳证据搜索与推理能力,以及原生工具调用在长视频场景中的有效性。

原文摘要 · Abstract (English)

Existing multimodal large language models for long-video understanding predominantly rely on uniform sampling and single-turn inference, limiting their ability to identify sparse yet critical evidence amid extensive redundancy. We introduce Video-o3, a novel framework that supports iterative discovery of salient visual clues, fine-grained inspection of key segments, and adaptive termination once sufficient evidence is acquired. Technically, we address two core challenges in interleaved tool invocation. First, to mitigate attention dispersion induced by the heterogeneity of reasoning and tool-calling, we propose Task-Decoupled Attention Masking, which isolates per-step concentration while preserving shared global context. Second, to control context length growth in multi-turn interactions, we introduce a Verifiable Trajectory-Guided Reward that balances exploration coverage with reasoning efficiency. To support training at scale, we further develop a data synthesis pipeline and construct Seeker-173K, comprising 173K high-quality tool-interaction trajectories for effective supervised and reinforcement learning. Extensive experiments show that Video-o3 substantially outperforms state-of-the-art methods, achieving 72.1% accuracy on MLVU and 46.5% on Video-Holmes. These results demonstrate Video-o3's strong multi-hop evidence-seeking and reasoning capabilities, and validate the effectiveness of native tool invocation in long-video scenarios.

视频推理多跳搜索工具调用长视频理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。