让大模型通过视觉变换任务自监督学习,提升视频理解能力
ViSS-R1: Self-Supervised Reinforcement Video Reasoning
- 用自监督强化学习让模型处理视觉变换任务,激发深层视觉理解
- 在6个基准上显著超越现有方法,有效减少幻觉和捷径学习
- 适合研究多模态大模型、视频理解与自监督学习的学者使用
复杂视频推理仍是多模态大语言模型(MLLM)的重大挑战,当前基于R1的方法多依赖文本中心推理,忽视丰富视觉信息,易导致捷径学习和幻觉。为此,我们提出一种新型自监督强化学习算法Pretext-GRPO,通过在变换后的视觉输入上正确解决预训练任务来获得正奖励,促使模型非平凡地处理视觉信息。在此基础上,我们构建了ViSS-R1框架,将基于预训练任务的自监督学习直接融入MLLM的R1后训练范式。该框架要求模型同时处理关于变换的预训练问题和真实用户查询,需识别施加的变换并重建原始视频以生成准确答案。在六个主流视频推理与理解基准上的全面评估表明,Pretext-GRPO与ViSS-R1在复杂视频推理中均表现出色且优于现有方法。代码与模型将公开。
原文摘要 · Abstract (English)
Complex video reasoning remains a significant challenge for Multimodal Large Language Models (MLLMs), as current R1-based methodologies often prioritize text-centric reasoning derived from text-based and image-based developments. In video tasks, such strategies frequently underutilize rich visual information, leading to potential shortcut learning and increased susceptibility to hallucination. To foster a more robust, visual-centric video understanding, we start by introducing a novel self-supervised reinforcement learning GRPO algorithm (Pretext-GRPO) within the standard R1 pipeline, in which positive rewards are assigned for correctly solving pretext tasks on transformed visual inputs, which makes the model to non-trivially process the visual information. Building on the effectiveness of Pretext-GRPO, we further propose the ViSS-R1 framework, which streamlines and integrates pretext-task-based self-supervised learning directly into the MLLM's R1 post-training paradigm. Instead of relying solely on sparse visual cues, our framework compels models to reason about transformed visual input by simultaneously processing both pretext questions (concerning transformations) and true user queries. This necessitates identifying the applied transformation and reconstructing the original video to formulate accurate final answers. Comprehensive evaluations on six widely-used video reasoning and understanding benchmarks demonstrate the effectiveness and superiority of our Pretext-GRPO and ViSS-R1 for complex video reasoning. Our codes and models will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。