arXiv:2603.28618cs.AI2026-03被引 7

让视觉理解和推理协同进化,提升多模态模型答对率

Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning

  • 用观察者和求解器双角色分工,分别负责提取证据和作答
  • 观察者通过求解器的成功率获得奖励,精准提升视觉理解
  • 在8个基准上平均提升7+点,适合想优化多模态推理的开发者

基于可验证奖励的强化学习(RLVR)显著提升了多模态大语言模型(MLLMs)的推理能力。然而现有方法通常依赖以最终答案为导向的共享奖励,同时更新感知与推理,导致信用分配模糊,常改善推理模式但无法可靠提升上游视觉证据提取的准确性。为解决这一感知瓶颈,我们提出双角色RLVR框架PRCO(Perception-Reasoning Coevolution),采用共享策略。PRCO包含两个协作角色:观察者生成与问题匹配的证据描述,求解器基于该描述预测最终答案。关键在于角色专用奖励:求解器使用可验证的最终答案奖励进行优化,观察者则根据求解器的下游成功率获得效用奖励。在八个挑战性多模态推理基准上的实验表明,PRCO在不同模型规模下均实现稳定提升,平均准确率超过基线模型7个百分点,优于先前开源的RL微调基线。

原文摘要 · Abstract (English)

Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR approaches typically rely on outcome-driven optimization that updates both perception and reasoning using a shared reward based solely on the final answer. This shared reward blurs credit assignment, frequently improving reasoning patterns while failing to reliably enhance the accuracy of upstream visual evidence extraction. To address this perception bottleneck, we introduce PRCO (Perception-Reasoning Coevolution), a dual-role RLVR framework with a shared policy. PRCO consists of two cooperative roles: an Observer that generates an evidence caption tailored to the question and a Solver that predicts the final answer based on this caption. Crucially, PRCO employs role-specific reward signals: the Solver is optimized using verifiable outcome rewards on the final answer, while the Observer receives a utility reward derived from the Solver's downstream success. Extensive experiments across eight challenging multimodal reasoning benchmarks demonstrate that PRCO yields consistent improvements across model scales by over 7 points on average accuracy compared to the base model, outperforming prior open-source RL-tuned baselines.

多模态推理强化学习视觉理解协同进化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。