让语言描述与图像实例精准对应，提升复杂指代分割准确率
Phrase-Instance Alignment for Generalized Referring Segmentation
- 将指代分割建模为实例级推理，先生成带语义的查询对象
- 通过细粒度对齐损失提升语言短语与视觉实例的匹配精度
- 可解释性强，适用于无目标或多个目标的复杂指代场景
广义指代表达可描述单一对象、多个相关对象或无目标。现有广义指代分割（GRES）模型对所有情况一视同仁，仅预测单个二值掩码，忽略语言短语与不同视觉实例之间的对应关系。为此，我们将GRES重新定义为实例级推理问题：模型首先根据指代表达生成多个感知实例的对象查询，再将每个查询与其最相关的短语对齐。该对齐通过短语-对象对齐（POA）损失实现，建立语言短语与视觉实例间的细粒度对应关系。基于这些对齐的实例查询及其学习到的相关性分数，最终分割结果和无目标情况均通过统一的相关性加权聚合机制推断得出。该实例感知框架实现了显式的短语-实例定位、可解释的推理过程，并能稳健处理复杂或空目标表达。在gRefCOCO和Ref-ZOM基准上的大量实验表明，本方法显著提升当前最优性能，达3.22% cIoU和12.25% N-acc。
原文摘要 · Abstract (English)
Generalized Referring expressions can describe one object, several related objects, or none at all. Existing generalized referring segmentation (GRES) models treat all cases alike, predicting a single binary mask and ignoring how linguistic phrases correspond to distinct visual instances. To this end, we reformulate GRES as an instance-level reasoning problem, where the model first predicts multiple instance-aware object queries conditioned on the referring expression, then aligns each with its most relevant phrase. This alignment is enforced by a Phrase-Object Alignment (POA) loss that builds fine-grained correspondence between linguistic phrases and visual instances. Given these aligned object instance queries and their learned relevance scores, the final segmentation and the no-target case are both inferred through a unified relevance-weighted aggregation mechanism. This instance-aware formulation enables explicit phrase-instance grounding, interpretable reasoning, and robust handling of complex or null expressions. Extensive experiments on the gRefCOCO and Ref-ZOM benchmarks demonstrate that our method significantly advances state-of-the-art performance by 3.22% cIoU and 12.25% N-acc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。