通过语言锚点库显式分离语义与定位,提升复杂查询分割精度
AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation

- 构建语言锚点库,将语义推理与空间定位解耦
- 在ReasonSeg上达67.7% gIoU和68.1% cIoU,性能领先
- 适合需要精准理解复杂文本指令的视觉分割任务
推理分割要求模型将复杂的隐含文本查询转化为精确的像素级掩码。现有方法依赖单一分割标记<SEG>,其隐状态同时编码语义推理与空间定位,限制了模型对两者的显式分离。我们提出AnchorSeg,将推理分割重构为基于语言锚点库的结构化条件生成过程。不将所有信息压缩到单一嵌入中,而是构建有序查询库:捕捉中间语义状态的潜在推理标记,以及提供显式空间定位的分割锚标记。我们将空间条件建模为图像标记上的分解分布,其中锚查询决定定位信号,上下文查询提供语义调制。为连接标记级预测与像素级监督,提出令牌-掩码循环一致性(TMCC)双向训练目标,强制跨分辨率对齐。通过结构化语言锚点库显式解耦空间定位与语义推理,AnchorSeg在ReasonSeg测试集上达到67.7% gIoU和68.1% cIoU的最先进水平。所有代码与模型公开于https://github.com/rui-qian/AnchorSeg。
原文摘要 · Abstract (English)
Reasoning segmentation requires models to ground complex, implicit textual queries into precise pixel-level masks. Existing approaches rely on a single segmentation token $\texttt{<SEG>}$, whose hidden state implicitly encodes both semantic reasoning and spatial localization, limiting the model's ability to explicitly disentangle what to segment from where to segment. We introduce AnchorSeg, which reformulates reasoning segmentation as a structured conditional generation process over image tokens, conditioned on language grounded query banks. Instead of compressing all semantic reasoning and spatial localization into a single embedding, AnchorSeg constructs an ordered sequence of query banks: latent reasoning tokens that capture intermediate semantic states, and a segmentation anchor token that provides explicit spatial grounding. We model spatial conditioning as a factorized distribution over image tokens, where the anchor query determines localization signals while contextual queries provide semantic modulation. To bridge token-level predictions and pixel-level supervision, we propose Token--Mask Cycle Consistency (TMCC), a bidirectional training objective that enforces alignment across resolutions. By explicitly decoupling spatial grounding from semantic reasoning through structured language grounded query banks, AnchorSeg achieves state-of-the-art results on ReasonSeg test set (67.7\% gIoU and 68.1\% cIoU). All code and models are publicly available at https://github.com/rui-qian/AnchorSeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。