ReAgent-V通过实时奖励驱动多智能体,提升视频理解的推理能力与泛化性。
ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding
- 引入动态奖励机制,支持多视角迭代反思与自纠正。
- 在12个数据集上实现最高9.8%的推理性能提升。
- 轻量模块化设计,适合视频理解、推理增强等任务。
视频理解是动作识别、视频推理和机器人控制等任务的基础。早期基于大视觉语言模型(LVLM)的方法通常采用单次推理范式,缺乏动态反馈,限制了模型在复杂场景下的自我修正与适应能力。近期工作尝试通过引入奖励模型和强化学习来提升推理能力,或采用工具-智能体框架,但面临标注成本高、奖励信号无法捕捉实时推理状态、推理效率低等问题。为此,我们提出ReAgent-V,一种新型代理式视频理解框架,结合高效帧选择与推理过程中的实时奖励生成。这些奖励信号不仅通过多视角反思机制(保守、中立、激进视角)引导答案迭代优化,还支持高质量数据自动筛选,用于监督微调(SFT)、直接偏好优化(DPO)和组相对策略优化(GRPO)。ReAgent-V轻量、模块化且可扩展,支持灵活工具集成。在12个数据集上的实验表明,其在视频理解、视频推理增强及视觉-语言-动作模型对齐三大核心应用中分别实现最高6.9%、2.1%和9.8%的性能提升,验证了框架的有效性与通用性。
原文摘要 · Abstract (English)
Video understanding is fundamental to tasks such as action recognition, video reasoning, and robotic control. Early video understanding methods based on large vision-language models (LVLMs) typically adopt a single-pass reasoning paradigm without dynamic feedback, limiting the model's capacity to self-correct and adapt in complex scenarios. Recent efforts have attempted to address this limitation by incorporating reward models and reinforcement learning to enhance reasoning, or by employing tool-agent frameworks. However, these approaches face several challenges, including high annotation costs, reward signals that fail to capture real-time reasoning states, and low inference efficiency. To overcome these issues, we propose ReAgent-V, a novel agentic video understanding framework that integrates efficient frame selection with real-time reward generation during inference. These reward signals not only guide iterative answer refinement through a multi-perspective reflection mechanism-adjusting predictions from conservative, neutral, and aggressive viewpoints-but also enable automatic filtering of high-quality data for supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). ReAgent-V is lightweight, modular, and extensible, supporting flexible tool integration tailored to diverse tasks. Extensive experiments on 12 datasets across three core applications-video understanding, video reasoning enhancement, and vision-language-action model alignment-demonstrate significant gains in generalization and reasoning, with improvements of up to 6.9%, 2.1%, and 9.8%, respectively, highlighting the effectiveness and versatility of the proposed framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。