arXiv:2501.05072cs.IRcs.CV2025-01

提出可扩展的视频片段检索框架,支持长视频多段匹配。

A Flexible and Scalable Framework for Video Moment Search

  • 分三阶段处理:段落检索、提案生成、精细重排
  • 在TVR-Ranking上达顶尖性能,耗时大幅降低
  • 适合大规模视频库应用,各阶段可独立优化

视频片段搜索旨在从视频库中定位与用户查询匹配的相关片段,对多种应用至关重要。现有方法通常假设仅存在单一完美匹配片段,推理效率低,且难以处理长达数小时的视频。本文提出一种灵活可扩展的框架,用于从任意长度视频集合中检索文本查询对应的排序片段列表,该任务称为排序视频片段检索(RVMR)。所提框架SPR将搜索过程分解为三个独立阶段:段落检索、提案生成与片段精炼重排。视频被划分为等长段落,其嵌入向量离线预计算并索引,实现与视频长度无关的高效检索。在线检索时,通过共享特征空间投影,使段落与查询均可进行近似最近邻(ANN)搜索,提升可扩展性。检索到的段落合并为粗粒度片段提案,再通过精炼与重排模块调整时间戳并重新排序。在TVR-Ranking数据集上的评估表明,该框架达到当前最优性能,同时显著降低计算成本与处理时间。灵活设计允许各阶段独立优化,适用于大规模应用场景。

原文摘要 · Abstract (English)

Video moment search, the process of finding relevant moments in a video corpus to match a user's query, is crucial for various applications. Existing solutions, however, often assume a single perfect matching moment, struggle with inefficient inference, and have limitations with hour-long videos. This paper introduces a flexible and scalable framework for retrieving a ranked list of moments from collection of videos in any length to match a text query, a task termed Ranked Video Moment Retrieval (RVMR). Our framework, called Segment-Proposal-Ranking (SPR), simplifies the search process into three independent stages: segment retrieval, proposal generation, and moment refinement with re-ranking. Specifically, videos are divided into equal-length segments with precomputed embeddings indexed offline, allowing efficient retrieval regardless of video length. For scalable online retrieval, both segments and queries are projected into a shared feature space to enable approximate nearest neighbor (ANN) search. Retrieved segments are then merged into coarse-grained moment proposals. Then a refinement and re-ranking module is designed to reorder and adjust timestamps of the coarse-grained proposals. Evaluations on the TVR-Ranking dataset demonstrate that our framework achieves state-of-the-art performance with significant reductions in computational cost and processing time. The flexible design also allows for independent improvements to each stage, making SPR highly adaptable for large-scale applications.

视频搜索检索框架长视频

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。