通过群体修订提升视觉语言模型在困难场景下的物体定位能力
From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Grounding
- 采用多轮修订生成候选答案，动态评估改进程度
- 在多个基准测试中显著优于传统GRPO方法
- 适合需要精准物体定位的复杂视觉理解任务
使用强化学习微调大型视觉-语言模型已成为提升其物体级定位能力的有前景方法。然而，现有方法主要基于GRPO，仅在响应层面分配奖励。这种稀疏奖励常由标准判定，当所有候选回答在挑战性场景中均失败时，学习信号极为微弱。本文提出一种群体修订优化范式，以增强对困难案例的学习。该方法从采样初始回答出发，生成一组修订候选，探索更优的定位结果。受奖励塑造启发，引入整合过程，量化每个候选相对于初始尝试的改进程度，并转化为具有信息量的塑造信号。这些信号既用于优化奖励，也调节优势值，放大高质量修订的影响。相比先前基于GRPO的模型，本方法在指代分割、推理分割、REC及计数等基准上实现持续性能提升。代码已开源：https://github.com/yyliu01/GroupRevision。
原文摘要 · Abstract (English)
Finetuning Large Vision-Language Models with reinforcement learning has emerged as a promising approach to enhance their capability in object-level grounding. However, existing methods, mainly based on GRPO, assign rewards at the response level. Such sparse reward, often criterion-induced, leads to minimal learning signals when all candidate responses fail in challenging scenarios. In this work, we propose a group-revision optimisation paradigm that enhances learning on hard cases. It begins with a sampled initial response and generates a set of revised candidates to explore improved grounding outcomes. Inspired by reward shaping, we introduce a consolidation process that quantifies each candidate's improvement over the initial attempt and converts it into informative shaping signals. These signals are used to both refine the reward and modulate the advantage, amplifying the influence of high-quality revisions. Our method achieves consistent gains across referring and reasoning segmentation, REC, and counting benchmarks compared with prior GRPO-based models. Our code is available at https://github.com/yyliu01/GroupRevision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。