让多模态大模型学会拒绝不存在的对象,提升可靠性。
Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO

- 采用校准强化学习,平衡定位准确与拒绝能力。
- 在三个基准上定位准确率超现有方法,拒绝率也更高。
- 适合需要高可靠性的视觉理解场景,如医疗、自动驾驶。
我们针对尚未充分探索的广义指代表达理解(GREC)任务,要求模型在对象存在时准确定位(正样本),在对象不存在时拒绝输出(负样本)。尽管多模态大语言模型(MLLMs)在定位已有对象方面表现优异,但因训练中缺乏负样本,常会生成虚构的边界框。现有后训练方法如监督微调(SFT)和强化学习(RL)虽提升了拒绝能力,却往往降低正样本的定位精度,损害模型核心性能。为此,我们提出拒绝校准的组相对策略优化(RC-GRPO),一种校准的强化学习策略,在强化拒绝能力的同时保持定位性能。它通过在负样本滚动生成中强制输出“无”以获得有效优势估计,并对正样本过度拒绝施加惩罚,实现准确率与可靠性的平衡。第二阶段推理强化进一步巩固因果理解与可解释性。在三个GREC基准上的实验表明,RC-GRPO在保持强拒绝能力的同时,实现了更优的定位准确率。
原文摘要 · Abstract (English)
We tackle the challenging yet underexplored task of Generalized Referring Expression Comprehension (GREC), which requires a model to localize the object described by a textual expression when it exists (positive sample) and to refuse output when it does not (negative sample). Although Multimodal Large Language Models (MLLMs) excel at localizing existing objects, they often fail to reject nonexistent ones due to the absence of negative samples during training, producing hallucinated bounding boxes. Existing post-training approaches such as supervised fine-tuning (SFT) and reinforcement learning (RL) enhance refusal behavior but usually degrade localization accuracy on positive samples, undermining the model's core competence. To address this, we propose Refusal-Calibrated Group Relative Policy Optimization (RC-GRPO), a calibrated RL strategy that strengthens the refusal ability of MLLMs while preserving localization performance. It enforces "None" outputs in rollouts for valid advantage estimation on negative samples and applies a penalty to prevent over-refusal on positives, achieving a balanced trade-off between accuracy and reliability. A second-stage reasoning reinforcement further consolidates causal understanding and interpretability. Experiments on three GREC benchmarks demonstrate that RC-GRPO attains superior localization accuracy while maintaining strong refusal capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。