arXiv:2508.15439cs.CV2025-08ICCV被引 1

用视频查询精准定位目标视频中的对应时刻,性能显著超越现有方法。

Aligning Moments in Time using Video Queries

  • 基于双阶段时序对齐的Transformer模型,捕捉语义与时间细节关联
  • 在ActivityNet-VRL上R@1提升13.1%,mIoU提升8.1%;SportsMoments上提升更明显
  • 适用于视频内容检索、动作定位,尤其适合需要跨视频匹配的任务

视频到视频时刻检索(Vid2VidMR)旨在利用查询视频定位目标视频中未见的事件或时刻。该任务面临语义帧级对齐和复杂视频间依赖建模的挑战。为此,我们提出MATR(Moment Alignment TRansformer),一种基于Transformer的模型,通过双阶段序列对齐机制,将查询视频特征条件化于目标视频表示,以编码所需的相关性和依赖关系。这些表示用于引导前景/背景分类和边界预测头,实现对目标视频中语义匹配时刻的精确识别。此外,为提供强任务初始化,我们设计了一种自监督预训练方法,让模型学习定位视频内随机片段。大量实验表明,MATR在主流ActivityNet-VRL数据集上相较先进方法绝对提升13.1%(R@1)和8.1%(mIoU);在新提出的SportsMoments数据集上,相比强基线分别提升14.7%(R@1)和14.4%(mIoU)。

原文摘要 · Abstract (English)

Video-to-video moment retrieval (Vid2VidMR) is the task of localizing unseen events or moments in a target video using a query video. This task poses several challenges, such as the need for semantic frame-level alignment and modeling complex dependencies between query and target videos. To tackle this challenging problem, we introduce MATR (Moment Alignment TRansformer), a transformer-based model designed to capture semantic context as well as the temporal details necessary for precise moment localization. MATR conditions target video representations on query video features using dual-stage sequence alignment that encodes the required correlations and dependencies. These representations are then used to guide foreground/background classification and boundary prediction heads, enabling the model to accurately identify moments in the target video that semantically match with the query video. Additionally, to provide a strong task-specific initialization for MATR, we propose a self-supervised pre-training technique that involves training the model to localize random clips within videos. Extensive experiments demonstrate that MATR achieves notable performance improvements of 13.1% in R@1 and 8.1% in mIoU on an absolute scale compared to state-of-the-art methods on the popular ActivityNet-VRL dataset. Additionally, on our newly proposed dataset, SportsMoments, MATR shows a 14.7% gain in R@1 and a 14.4% gain in mIoU on an absolute scale over strong baselines.

视频检索时序对齐Transformer动作定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。