将语义与几何信息分离，提升推理分割精度与可解释性。
SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation
- 设计语义-几何解耦提示框架，分别处理内容识别与空间定位。
- 在八组RefCOCO数据上达82.7%平均cIoU，ReasonSeg测试集达59.6% gIoU。
- 仅微调0.38%参数，适合资源受限场景下的视觉语言模型优化。
推理分割需将隐含语言结论转化为精确掩码，要求同时具备语义识别与空间定位能力。现有多模态大模型-分割器接口要么使用特殊触发词，要么将两种信号压缩至同一上下文，导致接收不同监督且失败模式各异，难以区分是理解错误还是定位失误。本文提出SeGDeP，一种显式的‘是什么-在哪里’接口：语义提示分支与独立几何投影路径分别将多模态大模型状态转换为语义特征与DETR预测框，共同引导SAM3掩码解码器。训练先对齐该可执行接口，再采用组奖励解耦策略优化（GDPO）平衡格式、框交并比（box-IoU）与掩码交并比（mask-IoU）反馈。SeGDeP-4B在八个RefCOCO系列分割任务中达到82.7%平均cIoU，ReasonSeg验证/测试集分别取得66.0%/59.6% gIoU，仅通过LoRA适配0.38%的Qwen3-VL参数。控制性分阶段消融实验、梯度诊断与提示干预进一步表明，两条路径发展出互补的语义与几何专长，而非重复同一证据。
原文摘要 · Abstract (English)
Reasoning segmentation converts an implicit linguistic conclusion into a precise mask, requiring both semantic identification and spatial grounding. Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context, although they receive different supervision and fail differently. This coupling obscures whether a failure arises from target interpretation or from localization. We present SeGDeP, an explicit what-where interface. A semantic prompt branch and an independent geometric projection path transform resolved MLLM states into semantic features and a DETR-predicted box, which jointly condition a SAM 3 mask decoder. Training first aligns this executable interface, then uses group reward-decoupled policy optimization (GDPO) to balance format, box-IoU, and mask-IoU feedback. SeGDeP-4B reaches 82.7 average cIoU over eight RefCOCO-family splits and 66.0/59.6 gIoU on ReasonSeg val/test while adapting only 0.38% of Qwen3-VL parameters through LoRA. Controlled stage-wise ablations, gradient diagnostics, and prompt interventions further show that the two paths develop complementary semantic and geometric specialization rather than duplicating the same evidence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。