提升视频语言模型的时间定位能力，用强化学习更高效找到准确时间片段。
TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs
- 用真实标注作为离策略监督，弥补传统方法在大时间空间中的搜索缺陷。
- 在多个数据集上刷新纪录，最高提升5.3%，且少样本下仍表现稳定。
- 支持复杂推理与简单问答统一处理，适合实际应用中多样化查询场景。
本文提出 TempSamp-R1，一种用于提升多模态大语言模型（MLLM）在视频时间定位任务中性能的强化微调框架。现有强化学习方法如组相对策略优化（GRPO）依赖于同策略采样，但在具有巨大时间搜索空间的任务中效率低、效果差，难以找到精确时间片段。为此，TempSamp-R1 利用真实标注作为离策略监督，提供精准的时间指导，有效缓解同策略解的稀疏与错位问题。为稳定训练并降低奖励更新方差，该方法引入非线性软优势计算，通过非对称变换动态重塑奖励反馈。同时，采用混合思维链（CoT）训练范式，使单一模型可同时支持含推理与无推理的推理模式，实现对不同复杂度查询的高效响应。实验表明，TempSamp-R1 在基准数据集上超越基于 GRPO 的基线，取得新最佳性能：Charades-STA（[email protected]: 52.9%，+2.7%）、ActivityNet Captions（[email protected]: 56.0%，+5.3%）、QVHighlights（mAP: 30.0%，+3.0%）。此外，其在少样本条件下也展现出强泛化能力。代码已开源。
原文摘要 · Abstract (English)
This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework designed to improve the effectiveness of adapting multimodal large language models (MLLMs) to video temporal grounding tasks. We reveal that existing reinforcement learning methods, such as Group Relative Policy Optimization (GRPO), rely on on-policy sampling for policy updates. However, in tasks with large temporal search spaces, this strategy becomes both inefficient and limited in performance, as it often fails to identify temporally accurate solutions. To address this limitation, TempSamp-R1 leverages ground-truth annotations as off-policy supervision to provide temporally precise guidance, effectively compensating for the sparsity and misalignment in on-policy solutions. To further stabilize training and reduce variance in reward-based updates, TempSamp-R1 provides a non-linear soft advantage computation method that dynamically reshapes the reward feedback via an asymmetric transformation. By employing a hybrid Chain-of-Thought (CoT) training paradigm, TempSamp-R1 optimizes a single unified model to support both CoT and non-CoT inference modes, enabling efficient handling of queries with varying reasoning complexity. Experimental results demonstrate that TempSamp-R1 outperforms GRPO-based baselines, establishing new state-of-the-art performance on benchmark datasets: Charades-STA ([email protected]: 52.9%, +2.7%), ActivityNet Captions ([email protected]: 56.0%, +5.3%), and QVHighlights (mAP: 30.0%, +3.0%). Moreover, TempSamp-R1 shows robust few-shot generalization capabilities under limited data. Code: https://github.com/HVision-NKU/TempSamp-R1
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。