arXiv:2505.20256cs.CV2025-05NeurIPS被引 37

用强化学习协同两个系统,解决多模态视频理解中分辨率与精度的矛盾。

Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration

  • 分两系统:全局推理选关键帧,细节理解做像素级定位。
  • 在RefAVS和REVOS上超越强基线模型,提升跨域泛化能力。
  • 首次实现大规模多模态推理的强化学习端到端训练,适合研究通用基础模型者。

长时序视频-音频推理与精细像素理解对多模态模型提出冲突需求:密集时间覆盖需大量低分辨率帧,而精准定位则依赖高分辨率输入。为此,我们提出双系统架构:全局推理系统以低成本选择信息性关键帧并重述任务,细节理解系统在选定的高分辨率片段上执行像素级定位。由于“最优”关键帧选择与任务重述难以明确标注,我们将其建模为强化学习(RL)问题,提出基于组相对策略优化的端到端框架Omni-R1。Omni-R1通过与细节理解系统在线协作获得层级奖励,仅需在小任务子集上进行单轮训练。在两个挑战性基准(Referring Audio-Visual Segmentation, RefAVS;Reasoning Video Object Segmentation, REVOS)上的实验表明,Omni-R1不仅超越强监督基线,还优于专用最先进模型,同时显著提升跨域泛化能力并缓解多模态幻觉。结果首次成功将强化学习应用于大规模多模态推理,揭示了通向通用基础模型的可扩展路径。

原文摘要 · Abstract (English)

Long-horizon video-audio reasoning and fine-grained pixel understanding impose conflicting requirements on omnimodal models: dense temporal coverage demands many low-resolution frames, whereas precise grounding calls for high-resolution inputs. We tackle this trade-off with a two-system architecture: a Global Reasoning System selects informative keyframes and rewrites the task at low spatial cost, while a Detail Understanding System performs pixel-level grounding on the selected high-resolution snippets. Because ``optimal'' keyframe selection and reformulation are ambiguous and hard to supervise, we formulate them as a reinforcement learning (RL) problem and present Omni-R1, an end-to-end RL framework built on Group Relative Policy Optimization. Omni-R1 trains the Global Reasoning System through hierarchical rewards obtained via online collaboration with the Detail Understanding System, requiring only one epoch of RL on small task splits. Experiments on two challenging benchmarks, namely Referring Audio-Visual Segmentation (RefAVS) and Reasoning Video Object Segmentation (REVOS), show that Omni-R1 not only surpasses strong supervised baselines but also outperforms specialized state-of-the-art models, while substantially improving out-of-domain generalization and mitigating multimodal hallucination. Our results demonstrate the first successful application of RL to large-scale omnimodal reasoning and highlight a scalable path toward universally foundation models.

多模态推理强化学习视频分割视觉语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。