零样本视频检索新框架,三步实现精准编辑后视频查找。
Reason, Retrieve, Re-rank: A Zero-Shot Reasoning-Aware Framework for Composed Video Retrieval
- 用大模型分析编辑后的状态变化,生成简洁描述
- 先粗筛再精排,最高达91.9%的R@1准确率
- 无需训练,适合快速部署于视频编辑场景
组合视频检索(CoVR)旨在找到对参考视频进行自由文本修改后生成的目标视频。本文针对CVPR 2026 VidLLMs研讨会提出的零样本感知推理挑战,提出完全由冻结基础模型构成的R3-CoVR(Reason, Retrieve, Re-rank)框架。多模态大语言模型Qwen3-VL-8B推断编辑带来的后效——如状态转移、动作阶段、场景、镜头与节奏,并生成简洁的后编辑描述;对比视频-文本编码器SigLIP-2将该描述与视频库嵌入以完成首轮检索;最后,约束感知重排序模块利用同一多模态模型作为裁判,对候选视频进行评分。在测试集上,R3-CoVR达到91.9% R@1和98.2% R@10。关键发现:(i) 将描述长度适配对比编码器文本窗口,使R@1从67.5提升至72.7;(ii) 约束重排序仅作用于短列表,使R@1从72.7跃升至91.9,为最大单次提升。论文分析了重排序行为、检索/重排序融合机制及短列表深度,并发布清晰的三层实现代码。
原文摘要 · Abstract (English)
Composed Video Retrieval (CoVR) seeks the target video that results from applying a free-form textual modification to a reference video. We address the \emph{Reason-Aware} CoVR (CoVR-R) challenge at the CVPR~2026 VidLLMs workshop, where retrieval is strictly zero-shot. We present \textbf{R3-CoVR} (\emph{Reason, Retrieve, Re-rank}), a training-free pipeline built entirely from frozen foundation models. A multimodal large language model (Qwen3-VL-8B) reasons about the \emph{after-effects} an edit implies -- state transitions, action phases, scene, camera and tempo -- and verbalises a concise post-edit description; a contrastive video--text encoder (SigLIP-2) embeds this description and the gallery for first-stage retrieval; finally a constraint-aware re-ranking stage uses the same multimodal model as a judge that scores each shortlisted candidate against the intended edited result. On the challenge test set, R3-CoVR attains \textbf{91.9\% R@1} and \textbf{98.2\% R@10}. Two findings drive these results: (i)~matching the description length to the contrastive encoder's text window lifts \Rk{1} from $67.5$ to $72.7$; and (ii)~the constraint-aware re-ranker, which reorders only the shortlist, lifts \Rk{1} from $72.7$ to $91.9$ -- the single largest gain. We analyse the re-ranker's behaviour, the retrieve/re-rank blend, and the shortlist depth, and we release a clean three-layer implementation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。