arXiv:2509.24786cs.CV2025-09被引 23

让视频模型会‘聚焦’,自动放大关键片段提升理解力

LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning

  • 通过多步推理动态调整视频采样分辨率,重点区域放大处理
  • 在4个基准上平均比基线高3.1个百分点,兼顾时序与细节
  • 适合需要精准理解长视频的场景,如纪录片分析、复杂事件追踪

长视频理解对当前大视频语言模型(LVLMs)仍具挑战,源于长时序理解与精细空间感知之间的矛盾。采用固定帧率和统一分辨率采样的模型,不可避免地牺牲时序信息或空间细节。为此,我们提出 LOVE-R1,一种可自适应聚焦视频片段的模型。模型初始接收密集采样但低分辨率的帧;若需更详细的空间信息,则基于推理能力主动放大关注片段的分辨率,直至获取关键视觉内容。该过程以多步推理实现。为训练推理能力,我们在自收集的38,000条高质量思维链数据上微调模型,并引入解耦强化微调。由于结果奖励无法提供细粒度过程监督,我们将多步推理分解为单步推理,显式优化内部聚焦能力。在多个长视频理解基准上的实验表明,采用慢-快自适应帧采样机制的模型,在采样密度与帧分辨率之间取得良好平衡,LOVE-R1 在4个常见长视频理解基准上平均比基线 Qwen2.5-VL 提升3.1个百分点。

原文摘要 · Abstract (English)

Long video understanding is still challenging for recent Large Video-Language Models (LVLMs) due to the conflict between long-form temporal understanding and detailed spatial perception. LVLMs with a uniform frame sampling mechanism, which samples frames with an equal frame size and fixed sampling rate, inevitably sacrifice either temporal clues or spatial details, resulting in suboptimal solutions. To mitigate this dilemma, we propose LOVE-R1, a model that can adaptively zoom in on a video clip. The model is first provided with densely sampled frames but in a small resolution. If some spatial details are needed, the model can zoom in on a clip of interest with a large frame resolution based on its reasoning until key visual information is obtained. The whole process is implemented as a multi-step reasoning process. To train the reasoning ability, we first finetune the model on our collected 38k high-quality CoT data and enhance it with decoupled reinforcement finetuning. As outcome rewards can not provide fine-grained process supervision, we decouple multi-step reasoning into multiple single-step reasoning and optimize the internal zoom-in ability explicitly. Experiments on long video understanding benchmarks show that our model with the slow-fast adaptive frame sampling mechanism achieves a great trade-off between sampling density and frame resolutions, and LOVE-R1 outperforms our baseline Qwen2.5-VL by an average of 3.1% points across 4 common long video understanding benchmarks.

长视频理解自适应采样多步推理视觉聚焦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。