arXiv:2507.07966cs.CVcs.AI2025-07NeurIPS被引 83

用强化学习让大模型读懂长视频,支持8000帧以上推理。

Scaling RL to Long Videos

  • 构建长视频问答数据集与两阶段训练框架,融合思维链与强化学习。
  • 模型在多个基准上准确率达65.1%~71.1%,支持8192帧视频处理。
  • 开源高效训练系统,单卡可训一小时长视频,适配多模态模型。

我们提出一个端到端框架,将视觉语言模型的推理能力扩展至长视频,利用强化学习实现。针对长视频推理挑战,集成三大组件:(1) 大规模数据集LongVideo-Reason,含10.4万条跨体育、游戏、vlog等领域的长视频问答对,附高质量推理标注;(2) 两阶段训练流程,先通过思维链监督微调(CoT-SFT),再引入强化学习(RL);(3) 面向长视频强化学习的训练架构MR-SP,结合序列并行与基于vLLM的引擎,使用缓存视频嵌入实现高效采样与预填充。实验显示,LongVILA-R1-7B在VideoMME上无字幕时达65.1%准确率,有字幕时达71.1%,优于LongVILA-7B。该模型支持每视频最多8192帧,可配置帧率。MR-SP系统在长视频强化学习训练中最高提速2.1倍。此外,我们开源训练系统,支持多种模态(视频、文本、音频)、多种模型(VILA、Qwen系列),甚至图像与视频生成模型。单个A100节点(8卡)可训练长达3,600帧(约一小时)的视频。

原文摘要 · Abstract (English)

We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 104K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In our experiments, LongVILA-R1-7B achieves strong performance on video benchmarks, reaching 65.1% and 71.1% accuracy on VideoMME without and with subtitles, respectively, and consistently outperforming LongVILA-7B across multiple benchmarks. Moreover, LongVILA-R1-7B supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP system achieves up to 2.1x speedup on long video RL training. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On a single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames).

长视频强化学习多模态推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。