用强化学习让模型自动生成可靠伪标签,提升少样本图像指代表达分割精度。
Learning to Label: A Reinforced Self-Evolving Framework for Semi-supervised Referring Expression Segmentation

- 将伪标签生成视为可学习的决策过程,结合多模态先验与文本线索构建引导信号。
- 在RefCOCO等数据集上显著优于现有方法,实现更稳定的像素级定位。
- 适合研究少样本视觉语言理解、自进化标注系统的研究者。
半监督指代表达分割(SS-RES)旨在有限标注下实现精确的像素级语言定位,但面临监督不足和未标注图像-文本对中伪标签不可靠的问题。本文提出「学习标注」(L2L)框架,将伪标签构建视为可学习的决策过程。首先利用多模态大语言模型提取语义-空间先验,生成初始软分割提议,并与文本线索融合为可学习的引导信号,指导分层分割网络。为确保稳定学习,采用强化学习机制进行伪标签选择,基于多模态先验和模型预测动态奖励高价值像素级监督。该强化自进化循环实现分割模型与伪标签的联合优化,在稀疏监督下逐步提升标签可靠性。在RefCOCO、RefCOCO+和RefCOCOg上的大量实验表明,该方法优于现有技术,验证了其有效性与泛化能力。
原文摘要 · Abstract (English)
Semi-supervised referring expression segmentation (SS-RES) aims to achieve precise pixel-level language grounding under limited annotation, yet suffers from limited supervision and unreliable pseudo-labels when exploiting unlabeled image-text pairs. In this work, we propose Learning to Label, a reinforced self-evolving framework (L2L) that casts pseudo-label construction as a learnable decision-making process. To build foundational understanding, we leverage a multimodal large language model to extract semantic-spatial priors, which are instantiated as initial soft segmentation proposals and elevated, together with textual cues, into learnable guidance signals that condition a hierarchical segmentation network. To ensure stable learning, reinforced pseudo-label selection is formulated as an exploratory decision process that adaptively rewards high-utility pixel-level supervision based on multimodal priors and model predictions. This reinforced self-evolving loop enables joint optimization of the segmentation model and pseudo-labels, progressively enhancing label reliability under sparse supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate improvements over existing methods, validating its effectiveness and generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。