arXiv:2510.12283cs.CV2025-10被引 1

提出新框架提升长视频中部分相关检索效果

Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrieval

  • 双分支学生网络从大模型迁移知识,分别继承与探索
  • 动态软目标机制让模型更好捕捉视频与查询的细粒度关联
  • 在多个数据集上达领先性能,适合真实长视频检索场景

以往文本到视频检索研究假设视频已预剪裁为短时长、内容单一的片段,但实际视频通常为长时长、背景复杂的未剪裁视频。为此,本文聚焦更贴近现实的局部相关视频检索(PRVR)任务,旨在根据查询检索部分相关的未剪裁视频。提出一种新型双学习框架DL-DKD++,通过一个强大的大规模视觉-语言预训练教师模型,将泛化知识动态蒸馏至轻量级、任务专用的学生网络。学生网络包含两个分支:继承分支吸收可迁移知识,探索分支从PRVR数据集中学习特定任务信息以缓解领域差异。进一步引入动态软目标构建机制,用随训练过程演变的自适应软标签替代固定硬标签,使模型更精准捕捉视频与查询间的细粒度部分相关性。实验表明,该方法在TVR、ActivityNet和Charades-STA数据集上均达到当前最优性能。代码已公开于https://github.com/HuiGuanLab/DL-DKD。

原文摘要 · Abstract (English)

Almost all previous text-to-video retrieval works ideally assume that videos are pre-trimmed with short durations containing solely text-related content. However, in practice, videos are typically untrimmed in long durations with much more complicated background content. Therefore, in this paper, we focus on the more practical yet challenging task of Partially Relevant Video Retrieval (PRVR), which aims to retrieve partially relevant untrimmed videos with the given query. To tackle this task, we propose a novel framework that distills generalization knowledge from a powerful large-scale vision-language pre-trained model and transfers it to a lightweight, task-specific PRVR network. Specifically, we introduce a Dual Learning framework with Dynamic Knowledge Distillation (DL-DKD++), where a large teacher model provides supervision to a compact dual-branch student network. The student model comprises two branches: an inheritance branch that absorbs transferable knowledge from the teacher, and an exploration branch that learns task-specific information from the PRVR dataset to address domain gaps. To further enhance learning, we incorporate a dynamic soft-target construction mechanism. By replacing rigid hard-target supervision with adaptive soft targets that evolve during training, our method enables the model to better capture the fine-grained, partial relevance between videos and queries. Experiment results demonstrate that our proposed model achieves state-of-the-art performance on TVR, ActivityNet, and Charades-STA datasets for PRVR. The code is available at https://github.com/HuiGuanLab/DL-DKD.

视频检索知识蒸馏弱监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。