用少量标注数据实现精准推理分割,提升模型泛化能力。
CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation
- 通过条件视觉指令捕捉物体间空间关系,增强语义理解
- 仅需100张标注图在Cityscapes上达到2.3%性能提升
- 适合标注稀缺场景的推理分割任务,如医学图像
推理分割旨在对复杂、常隐含指令中的目标生成像素级掩码,需基于场景上下文进行推理。尽管多模态语言模型已推动指令跟随分割发展,但泛化能力仍受限。主要瓶颈在于高质量像素标注与丰富语言监督配对的成本高昂,导致分布外性能脆弱。为此,我们提出CORA,一种半监督推理分割框架,联合利用少量标注数据与大量未标注图像。CORA引入三项核心组件:1)条件视觉指令,编码物体间的空间与上下文关系;2)基于多模态大模型在语义等价查询下输出一致性的噪声伪标签过滤机制;3)标签样本与伪标签样本间的令牌级对比对齐,增强特征一致性。这些设计使CORA在极低标注条件下实现鲁棒推理分割,在有限标注设置下超越现有基线。CORA在Cityscapes(城市场景理解基准)上仅需100张标注图像,性能领先基线+2.3%;在PanNuke(组织病理学数据集)上仅需180张标注图像,提升+2.4%,达当前最优水平。
原文摘要 · Abstract (English)
Reasoning segmentation seeks pixel-accurate masks for targets referenced by complex, often implicit instructions, requiring context-dependent reasoning over the scene. Recent multimodal language models have advanced instruction following segmentation, yet generalization remains limited. The key bottleneck is the high cost of curating diverse, high-quality pixel annotations paired with rich linguistic supervision leading to brittle performance under distribution shift. Therefore, we present CORA, a semi-supervised reasoning segmentation framework that jointly learns from limited labeled data and a large corpus of unlabeled images. CORA introduces three main components: 1) conditional visual instructions that encode spatial and contextual relationships between objects; 2) a noisy pseudo-label filter based on the consistency of Multimodal LLM's outputs across semantically equivalent queries; and 3) a token-level contrastive alignment between labeled and pseudo-labeled samples to enhance feature consistency. These components enable CORA to perform robust reasoning segmentation with minimal supervision, outperforming existing baselines under constrained annotation settings. CORA achieves state-of-the-art results, requiring as few as 100 labeled images on Cityscapes, a benchmark dataset for urban scene understanding, surpassing the baseline by $+2.3\%$. Similarly, CORA improves performance by $+2.4\%$ with only 180 labeled images on PanNuke, a histopathology dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。