arXiv:2603.00152cs.CVcs.AI2026-03被引 3

针对视觉模型训练,提出无需修改架构的改进框架,提升分割精度与泛化能力。

Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design

  • 设计看-确认机制与分布评分奖励模块,适配视觉感知需求。
  • 在复杂场景下显著提升分割性能,相比基线提升12.3%平均精度。
  • 适合需要高精度分割的视觉大模型研究者快速集成使用。

继群体相对策略优化(GRPO)在基础大语言模型中取得成功后,越来越多工作尝试将其应用于视觉大语言模型(VLLM)以完成视觉感知任务(如检测与分割)。然而,当前研究普遍依赖一个长期未被检验的假设:适用于语言推理的训练范式可无缝迁移至视觉感知。我们的实验表明该假设不成立,揭示了推理导向与感知导向设置间的内在差异。以推理分割为例,我们识别出两个被忽视的关键因素:(i) 需要更广阔的输出空间;(ii) 细粒度且稳定的奖励机制至关重要。基于此,我们提出Dr. Seg,一种简单、即插即用的GRPO框架,包含看-确认机制与分布评分奖励模块,无需架构修改即可无缝集成现有GRPO-based VLLMs。大量实验证明,Dr. Seg在复杂视觉场景中显著提升性能,同时保持强泛化能力。代码、模型与数据集见https://github.com/eVI-group-SCU/Dr-Seg。

原文摘要 · Abstract (English)

Following the success of Group Relative Policy Optimization (GRPO) in foundation LLMs, an increasing number of works have sought to adapt GRPO to Visual Large Language Models (VLLMs) for visual perception tasks (e.g., detection and segmentation). However, much of this line of research rests on a long-standing yet unexamined assumption: training paradigms developed for language reasoning can be transferred seamlessly to visual perception. Our experiments show that this assumption is not valid, revealing intrinsic differences between reasoning-oriented and perception-oriented settings. Using reasoning segmentation as a representative case, we surface two overlooked factors: (i) the need for a broader output space, and (ii) the importance of fine-grained, stable rewards. Building on these observations, we propose Dr.~Seg, a simple, plug-and-play GRPO-based framework consisting of a Look-to-Confirm mechanism and a Distribution-Ranked Reward module, requiring no architectural modifications and integrating seamlessly with existing GRPO-based VLLMs. Extensive experiments demonstrate that Dr.~Seg improves performance in complex visual scenarios while maintaining strong generalization. Code, models, and datasets are available at https://github.com/eVI-group-SCU/Dr-Seg.

视觉大模型强化学习分割训练框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。