从海量视频中自动找能对齐的片段，提升视频同步效率。
Sync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets
- 将视频对齐转为检索任务，用DRAQ指标筛选可对齐视频。
- 设计通用帧级特征，显著提升多种现有特征的对齐效果。
- 构建新评测基准，支持大规模视频对齐应用。
时间对齐旨在同步两段视频中的关键事件（如物体交互或动作阶段转换），有助于视频编辑、处理与理解。然而现有方法受限于必须提供合适的视频对，难以广泛应用。为此，本文将时间对齐重构为搜索问题，提出对齐视频检索（AVR）任务：给定查询视频，从大量剪辑中识别并同步最佳对齐视频。主要贡献包括：1）提出DRAQ，一种视频对齐度指标，用于候选视频的识别与重排序；2）设计一种有效且通用的帧级视频特征，显著提升多种现成特征表示的对齐性能；3）构建基于循环一致性度量的新基准与评估协议。在三个数据集（含大规模Kinetics700）上的实验表明，该方法能从多样化数据集中有效识别对齐视频对。
原文摘要 · Abstract (English)
Temporal video alignment aims to synchronize the key events like object interactions or action phase transitions in two videos. Such methods could benefit various video editing, processing, and understanding tasks. However, existing approaches operate under the restrictive assumption that a suitable video pair for alignment is given, significantly limiting their broader applicability. To address this, we re-pose temporal alignment as a search problem and introduce the task of Alignable Video Retrieval (AVR). Given a query video, our approach can identify well-alignable videos from a large collection of clips and temporally synchronize them to the query. To achieve this, we make three key contributions: 1) we introduce DRAQ, a video alignability indicator to identify and re-rank the best alignable video from a set of candidates; 2) we propose an effective and generalizable frame-level video feature design to improve the alignment performance of several off-the-shelf feature representations, and 3) we propose a novel benchmark and evaluation protocol for AVR using cycle-consistency metrics. Our experiments on 3 datasets, including large-scale Kinetics700, demonstrate the effectiveness of our approach in identifying alignable video pairs from diverse datasets. Project Page: https://daveishan.github.io/avr-webpage/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。