arXiv:2506.01725cs.CV2025-06被引 27

通过结构化思考增强多模态模型的视频描述能力,提升动作识别准确率。

VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking

  • 先分析视频主体属性与动作,再生成完整描述,引导模型深入推理。
  • 仅用1500样本训练,在多个数据集上动作准确率提升超6.9%。
  • 适合需要高精度动作描述的视频理解任务,如医疗或安防场景。

尽管强化学习在大语言模型推理能力提升中取得显著进展,但在视频字幕生成的多模态大模型中仍鲜有探索。本文首次系统研究基于GRPO的强化学习后训练方法在视频多模态大模型中的应用,旨在提升模型对视频动作的描述能力。我们提出VideoCap-R1,其通过结构化思考机制,先分析视频主体及其属性与动作,再生成完整字幕,并配备两个专用奖励机制:无需LLM的思维评分器评估推理质量,以及依赖LLM的字幕评分器评估输出质量。该强化学习框架有效建立了结构化推理与全面描述生成之间的联系,使模型能生成更精准的动作描述。实验表明,仅使用1500个样本,VideoCap-R1在多个视频字幕基准测试中均显著优于Qwen2VL-7B基线模型(DREAM1K:事件F1+4.4,VDC:准确率+4.2,CAREBENCH:动作F1+3.1,物体F1+6.9),且持续超越SFT训练模型,验证了GRPO在提升多模态模型字幕生成能力上的优越性。

原文摘要 · Abstract (English)

While recent advances in reinforcement learning have significantly enhanced reasoning capabilities in large language models (LLMs), these techniques remain underexplored in multi-modal LLMs for video captioning. This paper presents the first systematic investigation of GRPO-based RL post-training for video MLLMs, with the goal of enhancing video MLLMs' capability of describing actions in videos. Specifically, we develop the VideoCap-R1, which is prompted to first perform structured thinking that analyzes video subjects with their attributes and actions before generating complete captions, supported by two specialized reward mechanisms: a LLM-free think scorer evaluating the structured thinking quality and a LLM-assisted caption scorer assessing the output quality. The RL training framework effectively establishes the connection between structured reasoning and comprehensive description generation, enabling the model to produce captions with more accurate actions. Our experiments demonstrate that VideoCap-R1 achieves substantial improvements over the Qwen2VL-7B baseline using limited samples (1.5k) across multiple video caption benchmarks (DREAM1K: +4.4 event F1, VDC: +4.2 Acc, CAREBENCH: +3.1 action F1, +6.9 object F1) while consistently outperforming the SFT-trained counterparts, confirming GRPO's superiority in enhancing MLLMs' captioning capabilities.

视频字幕结构化思考强化学习多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。