提出一种高效文本视频检索方法,提升匹配精度同时降低计算开销。
Text-Video Retrieval With Global-Local Contrastive Consistency Learning

- 设计无参数的全局-局部交互模块,文本引导生成相关帧与视频特征。
- 引入对比一致性损失,增强正样本间得分一致性,抑制负样本混淆。
- 在三个数据集上表现优于现有方法,适合需要高效率检索的应用场景。
文本-视频检索旨在根据给定文本查询找到语义最相关的视频。然而,由于视频内容比文本更丰富多样,每个文本-视频对的主要语义常仅部分相关。现有方法多依赖语言-视频注意力模块实现对齐,虽有效但带来巨大计算开销,导致检索效率低下。本文提出一种简单而高效的方法——全局-局部对比一致性学习(GLCCL)。具体地,设计了一个无参数的全局-局部交互模块(GLIM),在文本引导下生成语义相关的帧级和视频级特征。此外,提出对比得分一致性(CSC)损失,促进正样本对间得分的一致性,抑制负样本对间的伪一致性。实验证明,CSC损失显著增强了模型对正样本与困难负样本的区分能力。在MSR-VTT、DiDeMo和VATEX三个基准数据集上的大量实验表明,该方法在效果和效率上均优于现有方法。
原文摘要 · Abstract (English)
Text-video retrieval aims to find the most semantically similar videos with given text queries. However, since videos contain more diverse content than texts, the main semantics expressed by each text-video pair is often partially relevant. The primary methods involve the utilization of language-video attention module to align texts and videos. Though effective, this paradigm inevitably introduces prohibitive computational overhead, resulting in inefficient retrieval. In this paper, we propose a simple yet effective method called Global-Local Contrastive Consistency Learning (GLCCL) to achieve texts and videos semantics alignment. Specifically, we design a parameter-free Global-Local Interaction Module (GLIM) to generate semantic-related frame and video features in a text-guided manner. Furthermore, a Contrastive Score Consistency (CSC) loss is developed to promote consistency learning among different scores on positive pairs and suppress consistency learning on negative pairs. Empirical evidence suggests that CSC loss provides the model with robust discriminative power between positives and hard negatives. Extensive experiments on three benchmark datasets, including MSR-VTT, DiDeMo and VATEX, demonstrate the effectiveness and superiority of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。