arXiv:2606.01113cs.CV2026-06被引 3

用推理引导召回与重排,精准检索编辑后的视频。

R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking

论文配图:R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking
图 1 · 摘自论文原文
  • 将编辑指令转为推理过程,生成目标视频的预期描述
  • 融合推理轨迹与源视频,提升召回准确率,优于基线18.7%
  • 适合视频编辑、跨模态检索研究者使用

CoVR-R挑战评估组合视频检索任务:给定参考视频和文本编辑指令,从大规模视频库中检索目标视频。该任务不同于标准视频-文本检索,查询由源视频的视觉信息与编辑所隐含的变换共同定义。强嵌入模型可实现高效候选召回,但可能忽略目标端的状态变化、动作替换、物体保留或时间一致性等细节。成对多模态重排器能更直接验证这些细节,但对全库进行穷举重排计算成本过高。本文提出$/mathbb{R}^3$,一种基于推理引导召回与重排的零样本组合视频检索框架。核心思想是将源-编辑查询转化为推理驱动的检索程序,而非将编辑文本视为简短描述。首先,模型生成描述应用编辑后目标视频的推理轨迹;随后,将该轨迹与源视频联合编码为推理增强查询,并通过一致门控残差规则融合基础查询得分;最后,重排器通过直接比较源视频与候选视频验证结果。实验表明该方法在应对该挑战上效果显著。代码已开源:https://github.com/Lee-zixu/R-3。

原文摘要 · Abstract (English)

The CoVR-R challenge evaluates composed video retrieval, where a system must retrieve a target video from a large gallery given a reference video and a textual edit instruction. This setting is not a standard video-text retrieval problem: the query is defined by both the visual evidence in the source video and the transformation implied by the edit. A strong embedding model can provide scalable candidate recall, but it may under-express target-side consequences such as state changes, action replacement, object preservation, or temporal consistency. A pairwise multimodal reranker can verify such details more directly, but exhaustive reranking over the full gallery is computationally infeasible. We present $\mathbb{R}^3$, a zero-shot composed video retrieval pipeline built around Reasoning-guided Recalling and Reranking. The core idea is to turn the source-edit query into a reasoning-grounded retrieval program rather than treating the edit text as a short caption. First, the model generates a reasoning trace that describes the expected target video after applying the edit. Then the trace is encoded together with the source video as a reasoning-augmented query, and its retrieval score is fused with the base composed query through an agreement-gated residual rule. At last, a re-ranker verifies the recalled candidates with direct source-candidate comparison. Experiments have demonstrated the effectiveness of our method in addressing this challenge. Codes are available on https://github.com/Lee-zixu/R-3.

视频检索推理引导零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。