arXiv:2510.11173cs.CVcs.MM2025-10中稿 · ICLR被引 9

用思维链生成可解释的位置先验,提升推理分割的精度与可读性。

CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation

  • 通过多模态思维链生成可微分的热力图作为位置先验
  • 在RefCOCO和ReasonSeg上达到或超过现有最佳性能
  • 适合需要可解释性与精准分割的研究者

现有推理分割方法要么直接将语言模型隐层特征连接到掩码解码器,要么仅表示文本位置,限制了可解释性和语义细节。为此,我们提出CoPRS,一种基于多模态思维链(MCoT)的位置感知模型,通过可微且可解释的位置先验热力图,将语言推理与分割相连接。该模型利用可学习的聚焦标记聚合图像与推理文本特征,生成稠密可微的热力图,并通过轻量解码器输出精确掩码,实现推理与分割的直接关联。在RefCOCO系列和ReasonSeg数据集上,CoPRS在标准划分下匹配或超越已有最优指标,验证了其在验证集和测试集上的领先表现。大量实验表明,思维链轨迹、生成热力图与解码掩码之间存在强正相关,支持推理输出与掩码生成之间的可解释对齐。结果证明该范式在推理与分割间的桥梁作用,展现由推理驱动的注意力集中与更精确掩码预测的优势。代码已开源:https://github.com/ZhenyuLU-Heliodore/CoPRS。

原文摘要 · Abstract (English)

Existing works on reasoning segmentation either connect hidden features from a language model directly to a mask decoder or represent positions in text, which limits interpretability and semantic detail. To solve this, we present CoPRS, a Multi-modal Chain-of-Thought (MCoT)-based positional perception model that bridges language reasoning to segmentation through a differentiable and interpretable positional prior instantiated as a heatmap. By making the reasoning process clear via MCoT and expressing it as a dense, differentiable heatmap, this interface enhances interpretability and diagnostic analysis and yields more concentrated evidence on the target. A learnable concentration token aggregates features of the image and reasoning text to generate this positional prior, which is decoded to precise masks through a lightweight decoder, providing a direct connection between reasoning and segmentation. Across the RefCOCO series and ReasonSeg, CoPRS matches or surpasses the best reported metrics on each standard split under comparable protocols, with performance at or above the prior state of the art across both validation and test partitions. Extensive experiments demonstrate a strong positive correlation among the CoT trajectory, the generated heatmap, and the decoded mask, supporting an interpretable alignment between the reasoning output and downstream mask generation. Collectively, these findings support the utility of this paradigm in bridging reasoning and segmentation and show advantages in concentration driven by reasoning and in more precise mask prediction. Code has been released at https://github.com/ZhenyuLU-Heliodore/CoPRS.

推理分割思维链可解释性热力图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。