arXiv:2603.27993cs.CV2026-03

通过分步推理提升图像中目标定位的准确性

Progressive Prompt-Guided Cross-Modal Reasoning for Referring Image Segmentation

  • 分三步逐步将语言描述转化为视觉定位:语义理解→空间定位→实例分割
  • 在RefCOCO+/RefCOCO++上平均mAP达68.7,优于现有方法
  • 适合需要精准语义-视觉对齐的应用场景,如智能图像编辑

指代图像分割旨在根据自由形式的指代表达,在图像中定位并分割目标物体。核心挑战在于有效连接语言描述与物体级视觉表征,尤其当指代表达包含详细属性和复杂的对象间关系时。现有方法或依赖跨模态对齐,或使用语义分割提示,但通常缺乏将语言描述显式地与图像目标区域关联的推理机制。为此,我们提出PPCR(Progressive Prompt-guided Cross-modal Reasoning)框架,明确将推理过程划分为语义理解-空间定位-实例分割的流水线。具体而言,首先利用多模态大语言模型(MLLMs)生成捕捉目标物体关键语义线索的语义分割提示;在此语义上下文中,进一步生成空间分割提示以推理目标位置与空间范围,实现从语义理解到空间定位的渐进过渡。最后,将语义与空间分割提示联合输入分割模块,引导精确的目标定位与分割。在标准指代图像分割基准测试上,大量实验证明PPCR持续优于现有方法。代码将公开发布,以促进可复现性。

原文摘要 · Abstract (English)

Referring image segmentation aims to localize and segment a target object in an image based on a free-form referring expression. The core challenge lies in effectively bridging linguistic descriptions with object-level visual representations, especially when referring expressions involve detailed attributes and complex inter-object relationships. Existing methods either rely on cross-modal alignment or employ Semantic Segmentation Prompts, but they often lack explicit reasoning mechanisms for grounding language descriptions to target regions in the image. To address these limitations, we propose PPCR, a Progressive Prompt-guided Cross-modal Reasoning framework for referring image segmentation. PPCR explicitly structures the reasoning process as a Semantic Understanding-Spatial Grounding-Instance Segmentation pipeline. Specifically, PPCR first employs multimodal large language models (MLLMs) to generate Semantic Segmentation Prompt that capture key semantic cues of the target object. Based on this semantic context, Spatial Segmentation Prompt are further generated to reason about object location and spatial extent, enabling a progressive transition from semantic understanding to spatial grounding. The Semantic and Spatial Segmentation prompts are then jointly integrated into the segmentation module to guide accurate target localization and segmentation. Extensive experiments on standard referring image segmentation benchmarks demonstrate that PPCR consistently outperforms existing methods. The code will be publicly released to facilitate reproducibility.

图像分割多模态推理提示工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。