用扩散模型生成视频全局语义标记，提升部分相关视频检索精度
Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval
- 先生成覆盖全视频的粗粒度语义标记，再精调跨模态匹配
- 在MSR-VTT和ActivityNet上分别提升3.2%和2.8%的Recall@10
- 适合需要精准定位局部事件的视频检索任务
部分相关视频检索（PRVR）旨在根据仅描述部分事件的文本查询，检索未剪辑视频。现有方法因全局上下文感知不完整，难以应对查询模糊性和由无关响应带来的局部噪声。为此，我们提出DreamPRVR，采用从粗到细的表示学习范式：首先基于视频中心分布初始化概率变分采样器，生成全局上下文语义标记；再通过文本监督的截断扩散模型迭代优化这些标记。在此过程中，文本语义结构学习构建了结构良好的文本潜在空间，增强了全局感知可靠性。随后，通过注册增强型高斯注意力模块，将标记自适应融合至视频标记，实现上下文感知的特征学习。大量实验表明，DreamPRVR优于当前最优方法。代码已开源：https://github.com/lijun2005/CVPR26-DreamPRVR。
原文摘要 · Abstract (English)
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。