用强化学习提升视觉模型跨图推理与定位能力
MIRG-RL: Multi-Image Reasoning and Grounding with Reinforcement Learning
- 分两阶段训练：监督微调+图像感知强化学习
- 跨图推理准确率达64.82%，比之前最优方法高1%
- 适合需要精准跨图理解的视觉问答与定位任务
多图推理与定位需同时理解物体级和图像级的复杂跨图关系。当前大视觉语言模型面临两大挑战：缺乏跨图推理能力，以及跨图指代奖励建模不足。为此，我们提出统一框架MIRG-RL，通过结合标注轨迹的监督微调与图像感知强化学习优化的两阶段训练范式，逐步构建多图推理能力。我们创新性地设计轨迹数据构建方法，融合物体级与图像级标注信息，生成轻量级增强推理数据集。为有效解决跨图歧义，设计双奖励函数的图像感知强化学习策略，分别针对物体与图像进行优化。实验表明，MIRG-RL在多图定位基准上达到当前最优性能，跨图推理任务准确率为64.82%，较之前最佳方法提升1%。代码与数据集已开源于https://github.com/ZEUS2035/MIRG-RL。
原文摘要 · Abstract (English)
Multi-image reasoning and grounding require understanding complex cross-image relationships at both object levels and image levels. Current Large Visual Language Models (LVLMs) face two critical challenges: the lack of cross-image reasoning capabilities and insufficient cross-image reference reward modeling. To address these issues, we propose a unified framework - Multi-Image Reasoning and Grounding with Reinforcement Learning (MIRG-RL). Specifically, our two-stage training paradigm combines supervised fine-tuning with annotated trajectories and image-aware reinforcement learning optimization, progressively developing multi-image reasoning capabilities. Furthermore, we innovatively propose a method for constructing the trajectory data, which integrates object-level and image-level annotation information, and use this method to generate a lightweight reasoning-enhanced dataset. To effectively resolve cross-image ambiguities, we design an image-aware RL policy with dual reward functions for objects and images. Experiments demonstrate that MIRG-RL achieves state-of-the-art (SOTA) performance in multi-image grounding benchmarks, attaining 64.82% on cross-image reasoning tasks - exceeding the previous best method by 1%. The code and dataset have been released at https://github.com/ZEUS2035/MIRG-RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。