arXiv:2607.04779cs.CV2026-07中稿 · ECCV被引 1

通过动态门控融合语义与空间提示,提升复杂语言查询下的分割精度。

DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation

论文配图:DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation
图 1 · 摘自论文原文
  • 分两路处理语义与空间提示,再用动态门控自适应融合预测结果。
  • 在ReasonSeg验证集和测试集上分别达到69.6%和67.3%的gIoU,超越基线。
  • 适合需要精准理解复杂语言指令的视觉推理分割任务。

推理分割旨在根据复杂的语言查询预测目标的像素级掩码。现有方法利用多模态大模型进行视觉-语言推理,并生成中间目标提示(如点或框)以引导分割模型。然而,将丰富推理压缩为稀疏提示常引入模糊性和噪声,难以准确保留推理意图。尽管多种互补提示可增强目标信息,但现有方法通常将它们联合输入单一分割过程,导致模糊或错误提示影响整体预测。为此,我们提出DGSeg,一种基于语义与空间提示动态门控融合的推理分割框架。具体地,多模态大模型同时推理目标身份与空间位置,生成互补的语义与空间提示,并分别输入独立的分割分支。其预测结果由轻量级动态门控模块自适应整合,该模块通过相对分支质量监督训练,抑制噪声或冲突区域。大量实验表明,DGSeg在多个基准上持续优于强基线,在挑战性的ReasonSeg验证集和测试集上分别取得69.6%和67.3%的gIoU。代码已开源。

原文摘要 · Abstract (English)

Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries. Existing approaches leverage Multimodal Large Language Models (MLLMs) for vision-language reasoning and generate intermediate target cues (e.g., points or boxes) to guide a segmentation model. However, compressing rich reasoning into sparse cues often introduces ambiguity and noise, preventing these cues from accurately preserving the reasoning intent. While multiple complementary cues can enrich target information, existing methods typically feed them jointly into a single segmentation process, allowing ambiguous or erroneous cues to affect the entire prediction. Therefore, we propose DGSeg, a reasoning segmentation framework that learns to fuse predictions guided by semantic and spatial cues. Specifically, the MLLM jointly reasons about both target identity and spatial location, producing complementary semantic and spatial cues that are fed into separate segmentation branches. Their predictions are adaptively integrated by a lightweight dynamic gating module trained with relative branch-quality supervision to suppress noisy or conflicting regions. Extensive experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks and achieves 69.6% and 67.3% gIoU on the challenging ReasonSeg validation and test splits. Code is available at https://github.com/RZZeng/DGSeg.

推理分割动态门控多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。