让视频大模型精准定位事件发生的时间段,支持多段、跨领域、多种提问方式。
TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

- 将时间定位视为区间集合,全程统一监督与优化。
- 在7个基准上超越所有同规模模型,8B版性能领先397B参数开源模型。
- 适合需要精确时间定位的视频理解任务,如智能搜索、内容编辑。
视频多模态大语言模型能描述视频内容,但很少识别支撑证据出现的时间。本文研究通用视频时间定位任务,即一个模型需预测跨视频长度、领域、查询形式和视角的可变数量证据区间。现有训练策略与该集合型任务不匹配:长视频标注常依赖脆弱的一次性标注,强化学习奖励或无法区分非重叠预测,或需易碎的片段匹配。TimeLens2在整个监督与优化过程中将时间证据视为区间集合。TimeLens2-93K通过字幕生成提议、独立定位、跨代理共识、语义验证和边界精炼构建可靠多跨度标注。其时间Wasserstein奖励计算合并区间支撑集上均匀分布间的一维$W_1$距离,提供稠密、免匹配的反馈,适用于不等数量与等效碎片化场景;时间IoU则补充精确重叠反馈。在七个基准上,TimeLens2-2B在每个任务上均超越所有同规模基线,4B和8B版本达到当前最佳性能,超越最大达397B参数的开源模型。2B、4B、8B版本分别较Qwen3-VL基线提升14.2、13.0、18.1 mIoU点。
原文摘要 · Abstract (English)
Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。