arXiv:2508.06317cs.CV2025-08NeurIPS被引 1

用少量无标签视频实现跨域视频定位,精度高且实时运行。

Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding

论文配图:Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding
图 1 · 摘自论文原文
  • 通过多轮推理生成伪标签,用置信度加权训练。
  • 仅需少量无标签目标域视频,跨域性能显著提升。
  • 适合无标注数据场景,支持实时系统部署。

视频时间定位(TG)旨在从长视频中定位与自然语言描述匹配的片段。尽管视觉语言模型(VLMs)在整体语义匹配上有效,但在细粒度时间定位上表现不佳。近期的组相对策略优化(GRPO)将推理过程建模为强化学习任务,实现了细粒度定位并在同域测试中表现优异。然而,GRPO依赖标注数据,难以适用于无标签域。此外,视频数据量大,全规模适应带来巨大延迟和计算开销,难以实现实时部署。为此,本文提出一种数据高效的无标签跨域时间定位方法:先在标注源域训练模型,再仅用少量目标域无标签视频进行适配。该方法无需目标域标注,且计算与存储开销极低,可实现实时运行。具体地,提出不确定性量化滚动策略适配(URPA),利用GRPO多轮推理生成多个候选预测,取平均作为伪标签,并通过方差估计置信度,以置信度加权训练奖励,引导模型关注可靠监督信号。在三个数据集、六个跨域设置上的实验表明,URPA仅用少量无标签视频即可实现良好泛化。代码将在发表后公开。

原文摘要 · Abstract (English)

Video Temporal Grounding (TG) aims to temporally locate video segments matching a natural language description (a query) in a long video. While Vision-Language Models (VLMs) are effective at holistic semantic matching, they often struggle with fine-grained temporal localisation. Recently, Group Relative Policy Optimisation (GRPO) reformulates the inference process as a reinforcement learning task, enabling fine-grained grounding and achieving strong in-domain performance. However, GRPO relies on labelled data, making it unsuitable in unlabelled domains. Moreover, because videos are large and expensive to store and process, performing full-scale adaptation introduces prohibitive latency and computational overhead, making it impractical for real-time deployment. To overcome both problems, we introduce a Data-Efficient Unlabelled Cross-domain Temporal Grounding method, from which a model is first trained on a labelled source domain, then adapted to a target domain using only a small number of unlabelled videos from the target domain. This approach eliminates the need for target annotation and keeps both computational and storage overhead low enough to run in real time. Specifically, we introduce. Uncertainty-quantified Rollout Policy Adaptation (URPA) for cross-domain knowledge transfer in learning video temporal grounding without target labels. URPA generates multiple candidate predictions using GRPO rollouts, averages them to form a pseudo label, and estimates confidence from the variance across these rollouts. This confidence then weights the training rewards, guiding the model to focus on reliable supervision. Experiments on three datasets across six cross-domain settings show that URPA generalises well using only a few unlabelled target videos. Codes will be released once published.

视频定位跨域迁移无监督学习强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。