arXiv:2505.15966cs.CVcs.AI2025-05NeurIPS被引 261

让视觉语言模型像人一样看图推理,提升复杂视觉任务准确率。

Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning

  • 给视觉模型添加缩放、选帧等图像操作,实现像素级直接推理。
  • 7B模型在多个基准上达84%准确率,为开源模型最高水平。
  • 用好奇心驱动强化学习平衡图文推理,解决模型不主动用图像操作的问题。

链式思维推理显著提升了大语言模型在多个领域表现,但其推理始终局限于文本空间,难以应对高视觉复杂度任务。为此,本文提出像素空间推理新范式:让视觉语言模型(VLMs)具备缩放、选帧等视觉操作能力,可直接观察、提问并推断视觉证据,从而增强视觉任务的推理准确性。训练面临模型初始能力不平衡及不愿使用新操作的挑战。我们采用两阶段训练:第一阶段通过合成推理轨迹进行指令微调,使模型熟悉新操作;第二阶段引入好奇心驱动的强化学习奖励机制,平衡像素空间与文本空间的探索。该方法使模型能主动处理信息密集的图像或视频,有效获取所需信息。实验表明,我们的7B模型在V* bench达到84%,TallyQA-Complex达74%,InfographicsVQA达84%,为当前开源模型最高成绩。结果验证了像素空间推理的重要性与框架的有效性。

原文摘要 · Abstract (English)

Chain-of-thought reasoning has significantly improved the performance of Large Language Models (LLMs) across various domains. However, this reasoning process has been confined exclusively to textual space, limiting its effectiveness in visually intensive tasks. To address this limitation, we introduce the concept of reasoning in the pixel-space. Within this novel framework, Vision-Language Models (VLMs) are equipped with a suite of visual reasoning operations, such as zoom-in and select-frame. These operations enable VLMs to directly inspect, interrogate, and infer from visual evidences, thereby enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space reasoning capabilities in VLMs presents notable challenges, including the model's initially imbalanced competence and its reluctance to adopt the newly introduced pixel-space operations. We address these challenges through a two-phase training approach. The first phase employs instruction tuning on synthesized reasoning traces to familiarize the model with the novel visual operations. Following this, a reinforcement learning (RL) phase leverages a curiosity-driven reward scheme to balance exploration between pixel-space reasoning and textual reasoning. With these visual operations, VLMs can interact with complex visual inputs, such as information-rich images or videos to proactively gather necessary information. We demonstrate that this approach significantly improves VLM performance across diverse visual reasoning benchmarks. Our 7B model, \model, achieves 84\% on V* bench, 74\% on TallyQA-Complex, and 84\% on InfographicsVQA, marking the highest accuracy achieved by any open-source model to date. These results highlight the importance of pixel-space reasoning and the effectiveness of our framework.

视觉推理强化学习多模态模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。