用反射边界标注和课程强化学习,让视频定位更准更快。
VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotations
- 通过预测标注外的相关时间点,识别并剔除不完整标注样本。
- 动态屏蔽难样本视频,在训练中逐步提升难度,提高学习效率。
- 仅用10%数据和21%算力,效果超越全量数据训练方法。
视频时空定位(VTG)旨在根据语言查询精确定位视频片段,是视频理解的基础挑战。尽管近期多模态大模型(MLLMs)通过强化学习(RL)在该任务上展现潜力,但其忽略了训练样本质量与难度带来的问题:(1)部分标注样本。许多样本包含标注区间外的相关片段,造成监督信号模糊;(2)难定位样本。零样本表现差的样本在RL训练中持续产生低且无差异的奖励,无法区分多个输出,阻碍学习效率。为此,我们提出VideoTG-R1,一种基于反射边界标注的课程强化学习框架,实现高效训练。具体地,提出边界反射代理,利用MLLM预测标注区间外的查询相关时间戳,从而识别并过滤部分标注样本,减少歧义。此外,设计难度评估代理,动态评估样本训练难度,并依据训练阶段对难样本视频进行掩码,缓解训练压力并提供更清晰偏好信号。在VTG与视觉问答任务上的实验表明,仅使用10%训练样本和21%计算预算,VideoTG-R1在组相对策略优化(GRPO)与监督微调(SFT)下均超越全量数据方法。
原文摘要 · Abstract (English)
Video temporal grounding (VTG) aims to locate precise segments in videos based on language queries, which is a fundamental challenge in video understanding. While recent Multimodal Large Language Models (MLLMs) have shown promise in tackling VTG through reinforcement learning (RL), they overlook the challenges arising from both the quality and difficulty of training samples. (1) Partially annotated samples. Many samples contain relevant segments beyond the annotated interval, introducing ambiguous supervision. (2) Hard-to-ground samples. Samples with poor zero-shot performance produce consistently low and indistinguishable rewards during RL training, exhibiting no clear preference among multiple outputs and thus hindering learning efficiency. To address these challenges, we propose VideoTG-R1, a novel curriculum RL framework with reflected boundary annotations, enabling data-efficient training. Specifically, we propose a Boundary Reflection Agent that utilizes MLLMs to predict query-relevant timestamps outside the annotated intervals, allowing us to identify and filter out partially annotated samples, thereby reducing ambiguity. Furthermore, we introduce a Difficulty Estimation Agent to assess the training difficulty of each sample and design a curriculum RL strategy that dynamically masks the videos of hard-to-ground samples according to the training steps, easing the training difficulty and providing clearer preference. Experiments on the VTG and grounded VideoQA tasks demonstrate the effectiveness of our method. Remarkably, with only 10% of the training samples and 21% of the computational budget, VideoTG-R1 outperforms full-data counterparts under both group relative policy optimization (GRPO) and supervised fine-tuning (SFT). The code is available at https://github.com/ldong1111/VideoTG-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。