用自评机制统一视觉区域理解与定位,无需标注文本。
Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO

- 让模型先生成区域描述,再自我校验定位准确性,形成闭环优化。
- 在多个任务上零样本提升,区域描述和定位性能同步增强。
- 适合希望提升多模态模型像素级能力的研究者与开发者。
本文提出一种统一的强化学习框架——循环组相对策略优化(CycleGRPO),用于联合优化多模态大语言模型(MLLM)的区域理解与定位能力。不同于传统分离式流水线,该方法利用两项任务间的内在对偶性,构建“区域→文本→区域”的自评估机制:单一MLLM先作为执行者生成区域描述,随即切换为批评者,将生成的文本回映到空间域进行验证。因此,该框架仅需区域输入(如掩码或边界框),完全无需文本真实标签。通过质量感知的词粒度循环一致性奖励,评估文本描述在物理定位上的语义区分度。基于SAMTok实现的实验表明,无需任何任务特定微调,该框架在区域描述、区域VQA、 grounded对话和指代分割等广泛基准上均取得一致性能提升。总体而言,CycleGRPO为推进MLLM的像素级能力提供了一种简洁且可扩展的方案。代码与模型已开源于https://github.com/devinxzhang/CycleGRPO。
原文摘要 · Abstract (English)
This paper introduces Actor as Its Own Critic, a unified reinforcement learning framework, Cycle Group Relative Policy Optimization (CycleGRPO), that jointly optimizes region understanding and localization for Multimodal Large Language Models (MLLMs). Unlike existing separate pipelines, we leverage the inherent duality between the two tasks to construct a self-evaluating reinforcement learning paradigm: "region $\to$ text $\to$ region''. Specifically, a single MLLM first acts as the actor to generate region captions, then immediately transitions to a critic to ground its generated text back in the spatial domain. Therefore, CycleGRPO requires only region inputs, e.g., masks or bounding boxes, entirely bypassing the need for textual ground truths. A quality-aware token-level cycle-consistency reward is employed to assess the semantic discriminability of text captions via their physical localization accuracy. Empirically, built upon SAMTok, our CycleGRPO framework successfully bootstraps both capabilities simultaneously. Without any task-specific fine-tuning, the framework yields consistent performance gains across a wide range of benchmarks, including region captioning, region VQA, grounded dialogue, and referring segmentation. Overall, CycleGRPO offers a straightforward and scalable way to advance pixel-level capabilities in MLLMs. Code and models are released at https://github.com/devinxzhang/CycleGRPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。