用强化学习统一优化分割模型,提升复杂指代分割的准确率。
B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation

- 提出GRTO框架,让分割器与策略协同优化
- B-GRTO预训练使收敛更快,性能超越基线方法
- 适合需要强推理能力的视觉语言分割任务
分割是计算机视觉的基础任务,支撑像素级场景理解,并广泛应用于自动驾驶和医学图像分析。针对复杂的指代分割问题,现有方法将大视觉语言模型与分割解码器结合:前者分析图像和提示,后者预测目标掩码。尽管强化学习能提升推理密集型视觉语言系统,但可训练工具(如分割解码器)通常通过可微目标单独优化,而如何将此类目标与强化学习有机结合仍缺乏系统研究。为此,本文提出群组相对工具优化(GRTO),一种数学严谨的联合优化框架,用于在不同可微工具使用中优化策略。GRTO复用群组相对策略优化(GRPO)的采样结果来优化辅助工具目标,使解码器梯度补充策略奖励。进一步推导出自举式GRTO(B-GRTO),一种低成本预训练方法,能快速启动工具优化,实现更快收敛与更优性能。在三个具有挑战性的指代分割设置中,B-GRTO显著优于普通GRPO,达到或超越特定领域最先进的方法。这证明了将强化学习与可微辅助目标统一对于推理密集型分割的价值。
原文摘要 · Abstract (English)
Segmentation is a fundamental task in computer vision, underpinning pixel-level scene understanding and serving as a cornerstone for applications ranging from autonomous perception to medical image analysis. For complex referring segmentation, recent methods pair large vision-language models with segmentation decoders: the former analyzes the image and prompt, while the latter predicts the target mask. Although reinforcement learning improves reasoning-intensive vision-language systems, trainable tools such as segmentation decoders are typically optimized separately with differentiable objectives, and the principled integration of such objectives into reinforcement learning remains underexplored. Thus, we introduce group relative tool optimization (GRTO), a mathematically grounded framework for jointly optimizing a policy with differentiable tool use. GRTO reuses group relative policy optimization (GRPO) rollouts to optimize the auxiliary tool objective, letting decoder gradients complement policy rewards. Further, we derive Bootstrapped-GRTO (B-GRTO), a pre-training method that cheaply bootstraps the tool, leading to faster convergence and superior performance. Across three challenging referring segmentation settings, B-GRTO results in substantial improvements over plain GRPO, matching or surpassing domain-specific state-of-the-art methods. This demonstrates the value of unifying reinforcement learning with differentiable auxiliary objectives for reasoning-intensive segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。