arXiv:2602.22740cs.CVcs.AI2026-02

通过过滤低对齐像素,提升图像指代分割的准确性与鲁棒性。

AMLRIS: Alignment-aware Masked Learning for Referring Image Segmentation

  • 基于视觉语言相似度动态屏蔽低对齐区域,聚焦可靠线索。
  • 在RefCOCO全系列数据集上达到当前最佳性能,8个分割任务均领先。
  • 无需修改模型结构,训练无额外开销,适合实际部署场景。

指代图像分割(RIS)旨在分割由自然语言描述唯一指代的图像对象。然而,传统RIS训练常包含难以对齐且实例特定的视觉信号;在这些像素上优化会引入误导性梯度,导致模型学习方向错误。本文提出一种对齐感知掩码学习(AML),通过量化区域-指代对齐度(PMME)并过滤不可靠像素(AFM),使模型专注于高对齐区域。具体地,每样本先计算视觉与文本特征的相似度图,再将低于自适应阈值的像素掩码掉,从而排除低对齐区域的训练参与。AML无需架构修改,训练无额外开销,能有效引导模型关注与文本描述一致的区域。在RefCOCO(vanilla/+/g)数据集上的实验表明,AML在全部8个划分上均达到最先进水平,不仅提升分割性能,还增强模型对多样描述和复杂场景的鲁棒性。代码已开源。

原文摘要 · Abstract (English)

Referring Image Segmentation (RIS) aims to segment the object in an image uniquely referred to by a natural language expression. However, RIS training often contains hard-to-align and instance-specific visual signals; optimizing on such pixels injects misleading gradients and drives the model in the wrong direction. By explicitly estimating pixel-level vision-language alignment, the learner can suppress low-alignment regions, concentrate on reliable cues, and acquire more generalizable alignment features. In this paper, we propose Alignment-Aware Masked Learning (AML), a simple yet effective training strategy that quantifies region-referent alignment (PMME) and filters out unreliable pixels during optimization (AFM). Specifically, each sample first computes a similarity map between visual and textual features, and then masks out pixels falling below an adaptive similarity threshold, thereby excluding poorly aligned regions from the training process. AML does not require architectural changes and incurs no inference overhead, directing attention to the areas aligned with the textual description. Experiments on the RefCOCO (vanilla/+/g) datasets show that AML achieves state-of-the-art results across all 8 splits, and beyond improving RIS performance, AML also enhances the model's robustness to diverse descriptions and scenarios. Code is available at https://github.com/pipashu1/AMLRIS.

图像分割视觉语言对齐掩码学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。