arXiv:2509.04773cs.CV2025-09ICCV被引 7

提出混合架构,让文本视频检索又快又准。

Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrieval

  • 为每个视频生成伪文本查询,实现细粒度特征交互。
  • 在5个数据集上R@1提升1.6%至3.9%,接近顶尖性能。
  • 推理效率媲美双塔模型,适合实际部署场景。

文本到视频检索(T2VR)旨在通过语义匹配的文本查询检索未标注视频。现有基于CLIP的方法采用双塔或单塔框架,前者效果差,后者效率低。本文提出新型混合塔框架,引入细粒度伪查询生成与交互方法(PIG),为每个视频生成伪查询,使视频特征与伪文本特征在细粒度层面交互,类似单塔结构以保持高效果,且在真实查询到达前即可完成。推理阶段不增加存储或计算开销,与双塔框架效率相当。在五个常用文本-视频检索基准上的实验证明,该方法相比基线在R@1上提升1.6%~3.9%。同时达到接近当前最优性能,且效率与双塔模型相当,凸显混合塔框架优势。

原文摘要 · Abstract (English)

The Text-to-Video Retrieval (T2VR) task aims to retrieve unlabeled videos by textual queries with the same semantic meanings. Recent CLIP-based approaches have explored two frameworks: Two-Tower versus Single-Tower framework, yet the former suffers from low effectiveness, while the latter suffers from low efficiency. In this study, we explore a new Hybrid-Tower framework that can hybridize the advantages of the Two-Tower and Single-Tower framework, achieving high effectiveness and efficiency simultaneously. We propose a novel hybrid method, Fine-grained Pseudo-query Interaction and Generation for T2VR, ie, PIG, which includes a new pseudo-query generator designed to generate a pseudo-query for each video. This enables the video feature and the textual features of pseudo-query to interact in a fine-grained manner, similar to the Single-Tower approaches to hold high effectiveness, even before the real textual query is received. Simultaneously, our method introduces no additional storage or computational overhead compared to the Two-Tower framework during the inference stage, thus maintaining high efficiency. Extensive experiments on five commonly used text-video retrieval benchmarks demonstrate that our method achieves a significant improvement over the baseline, with an increase of $1.6\% \sim 3.9\%$ in R@1. Furthermore, our method matches the efficiency of Two-Tower models while achieving near state-of-the-art performance, highlighting the advantages of the Hybrid-Tower framework.

文本视频检索伪查询生成高效检索混合架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。