ReTrack通过校准多模态特征方向偏差，提升视频检索精度。
ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrieval
- 分离语义贡献，校准视频与文本的特征方向偏差
- 利用双向证据实现精准的目标视频相似度估计
- 在视频与图像检索任务中均达到当前最优性能
随着视频数据快速增长，组合视频检索（CVR）作为一种新型视频检索范式受到广泛关注。与单模态检索不同，CVR以参考视频和修改文本构成的多模态查询为输入，目标是检索出符合用户意图变化的目标视频。然而，视频与文本在信息密度上存在显著差异，传统融合方法常使特征偏向参考视频，导致检索效果不佳。该问题源于三大挑战：模态贡献纠缠、组合特征显式优化缺失、检索不确定性。为此，本文提出证据驱动的双流定向锚点校准网络（ReTrack），首次通过校准组合特征的方向偏差来增强多模态查询理解。ReTrack包含三个核心模块：语义贡献解耦、组合几何校准、可靠证据驱动对齐。它先估计各模态的语义贡献，校准组合特征方向；再利用校准后的方向锚点生成双向证据，驱动可靠的组成-目标相似度计算。此外，ReTrack在组合图像检索（CIR）任务上也表现出强泛化能力，在三个基准数据集上均取得当前最优表现。代码已开源。
原文摘要 · Abstract (English)
With the rapid growth of video data, Composed Video Retrieval (CVR) has emerged as a novel paradigm in video retrieval and is receiving increasing attention from researchers. Unlike unimodal video retrieval methods, the CVR task takes a multi-modal query consisting of a reference video and a piece of modification text as input. The modification text conveys the user's intended alterations to the reference video. Based on this input, the model aims to retrieve the most relevant target video. In the CVR task, there exists a substantial discrepancy in information density between video and text modalities. Traditional composition methods tend to bias the composed feature toward the reference video, which leads to suboptimal retrieval performance. This limitation is significant due to the presence of three core challenges: (1) modal contribution entanglement, (2) explicit optimization of composed features, and (3) retrieval uncertainty. To address these challenges, we propose the evidence-dRivRn dual-sTream diRectionAl anChor calibration networK (ReTrack). ReTrack is the first CVR framework that improves multi-modal query understanding by calibrating directional bias in composed features. It consists of three key modules: Semantic Contribution Disentanglement, Composition Geometry Calibration, and Reliable Evidence-driven Alignment. Specifically, ReTrack estimates the semantic contribution of each modality to calibrate the directional bias of the composed feature. It then uses the calibrated directional anchors to compute bidirectional evidence that drives reliable composed-to-target similarity estimation. Moreover, ReTrack exhibits strong generalization to the Composed Image Retrieval (CIR) task, achieving SOTA performance across three benchmark datasets in both CVR and CIR scenarios. Codes are available at https://github.com/Lee-zixu/ReTrack
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。