让视频定位任务各司其职，提升精准度与效率
TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding
- 用专家路由机制区分时间、重要性、文本生成任务
- 在多个视频定位任务上达到当前最优效果
- 适合需要高精度视频片段识别的研究与应用
视频时间定位（VTG）旨在根据文本查询精准识别视频中的事件片段。输出为一系列事件，每个事件包含精确的时间戳、显著性分数和文本描述。尽管近期进展显著，现有视频大语言模型（Video-LLMs）仍存在根本局限：所有任务令牌均通过相同且静态的路径处理，未能区分时间定位、显著性评估和文本生成三类本质不同的任务，导致处理不专精。为此，我们提出TimeExpert，一种基于混合专家（MoE）架构的视频大语言模型，通过动态路由将特定任务令牌（如时间戳、显著性分数）分配给专用专家，实现任务解耦，同时提升计算效率。该设计使各类子任务得到更精准处理，在多种VTG应用中表现更优。大量实验表明，TimeExpert在密集视频描述、时刻检索和视频亮点检测等任务上持续达到当前最优性能。
原文摘要 · Abstract (English)
Video Temporal Grounding (VTG) aims to precisely identify video event segments in response to textual queries. The outputs of VTG tasks manifest as sequences of events, each defined by precise timestamps, saliency scores, and textual descriptions. Despite recent advances, a fundamental limitation persists in existing Video Large Language Models (Video-LLMs): they process all task tokens through identical and static pathways, failing to recognize that temporal localization, saliency assessment, and textual generation represent fundamentally distinct tasks requiring specialized processing. To address this, we introduce TimeExpert, a Mixture-of-Experts (MoE)-based Video-LLM that effectively decomposes VTG tasks by dynamically routing task-specific tokens (e.g., timestamps, saliency scores) to specialized experts, with increased computational efficiency. Our design choices enable precise handling of each subtask, leading to improved event modeling across diverse VTG applications. Extensive experiments demonstrate that TimeExpert consistently achieves state-of-the-art performance on various VTG tasks such as Dense Video Captioning, Moment Retrieval, and Video Highlight Detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。